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Across all industries the demands of data infrastructure have soared to new heights. 


As capacity requirements continue to rise at an ever-increasing rate, performance must not be compromised. The hybrid 
architecture and advanced software capabilities of the TrueNAS appliance enable users to be more agile, effectively 
manage the explosion of unstructured data and deploy a centralized information storage infrastructure. Whether it’s 
backing virtual machines, business applications, or web services, there’s a TrueNAS appliance suited to the task. 


TrueNAS™ Storage Appliances: Harness The Cloud 


iXsystems’ TrueNAS Appliances offer scalable high-throughput, low latency storage 


All TrueNAS Storage Appliances feature the Intel® Xeon® Processors 5600 series, powering the fastest data transfer 
speeds and lowest latency possible. TrueNAS appliances come in three lines: Performance, Archiver, & High Availability. 
High-performance, high-capacity ioMemory modules from Fusion-io are available in the TrueNAS Enterprise, Ultimate, 
and Archiver Pro models. 


Key Features: 


¢ One or Two Six-Core Intel® Xeon® Processors 
5600 series 


¢ Share Data over CIFS, NFS and iSCSI 

¢ Hybrid storage pool increases performance and 
decreases energy footprint 

¢ 128-bit ZFS file system with up to triple parity 
software RAID 
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Dear Kea@ers., 


in PC-BSD 9.1. So, you can check now what you a are waiting for. 

September’s Dev Corner is dedicated to PC-BSD. In the second 
article from this section Kris Moore will show you how to set up 
OwnCloud via the Warden. It received very good reviews from 
betatesters, so you should enjoy it much. 

We also introduced a new series by Rob Somerville. This 
time he will show you step by step how to build a search engine 
using Apache SOLR. A great grasp of practical knowledge — just 
as you like it. 

This month we launched The Best of BSD 2011. You will find 
there the best BSD Magazine articles of 2011 with updates. The 
idea is to sum up the 2011 year, not write a new one, so still you 
can find the references to old releases. It gives the opportunity to 
compare the past with the present and to follow the development 
of BSD systems and users’ needs. The other purpose of this issue 
is to support BSD Magazine, so it can maintain its position on the 
market as a free on-line magazine. 

You may buy the issue on: http://stackmag.org 


Wish you a good read! 
Patrycja Przybylowicz 
& BSD Team 
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Get Started 
OG Nmap: The Network Swiss Army Knife 


By Giovanni Bechis 
Nmap (“Network Mapper”) is a GPL utility for network dis- 
covery and security auditing. Many systems and network 
administrators find it very useful for network inventory, 
monitoring hosts and services uptime, debugging network 
related problems, and many other tasks. From this article 
you will learn the basic functionalities of Nmap 6. 


Developers Corner 


10 What's New in PC-BSD 9.1 
By Dru Lavigne 

PC-BSD 9.1 adds many new features, ranging from more 
graphical utilities available within Control Panel, a rede- 
signed installer, a server installation wizard, and improved 
jail management. This article introduces these new fea- 
tures. PC-BSD 9.1 is expected to be released during Sep- 
tember, 2012. This article introduces some of the new fea- 
tures of this release. 


4q, Setting up Your OwnCloud Instance 

via The Warden™ 

By Kris Moore 
In this article we will be taking a look at the OwnCloud 
software, specifically how to do the initial installation and 
configuration inside a jail run by PC-BSD’s® jail manage- 
ment utility, the Warden™. First we will take a look at a 
setup done from a PC-BSD graphical interface, and then 
explore the same setup from the command-line using 
TrueOS™, the server version of PC-BSD. 


How To 


18 Unix IPC with Pipes 
By Paul McMath 
This article explains one of the earliest forms of inter-pro- 
cess communication (IPC) in Unix. Pipes were the origi- 
nal form of Unix IPC and were present in Third Edition of 
Unix (1973). They can only be used to communicate 
between related processes, but despite this limita- 
tion they still remain one of the most frequently 
employed mechanisms for IPC. 


2 4 FreeBSD Enterprise Search with Apache 
Solr Part 1 
By Rob Somerville 
Back office integration and cross platform search has al- 
ways posed major challenges especially in large orga- 
nizations with many legacy systems. With Apache Solr 
these barriers can be overcome and the power of enter- 
prise search realised. In this new series the author will 
show you step by step how to commission an Apache Solr 
search engine. 


3 4 PostgreSQL Partitioning (Part 2) 
By Luca Ferrari 

In this article the readers will further extend the applica- 
tion scenario presented in the previous part, implement- 
ing a physical partitioning that keeps tables and data in 
separate storage devices. All the examples shown here 
have been tested on a PostgreSQL 9.1 cluster running on 
a FreeBSD 8.2-RELEASE machine; see the previous ar- 
ticle in this series for details about the application scenario 
and how to reproduce it. 


security 


4 QO Hardening FreeBSD with TrustedBSD 
and Mandatory Access Controls (MAC) 
Part 3 
By Michael Shirk 
Most system administrators understand the need to lock 
down permissions for files and applications. In addition to 
these configuration options on FreeBSD, there are fea- 
tures provided by TrustedBSD that add additional layers 
of specific security controls to fine tune the operating sys- 
tem for multilevel security. By reading this article you will 
learn the configuration of the mac_bsdextended module 
and how to use the ugidfw utility 


Interview 


4 Q Interview with 

Jeroen van Nieuwenhuizen 

By BSD Team 
Jeroen van Nieuwenhuizen was the chair of the EuroBS- 
Dcon 2011 organizing committee. Currently, he is one of 
the members of the EuroBSDcon Foundation board. He 
came in contact with Unix in 1997 and started to work with 
the BSDs in 2002. In his daily life Jeroen works as a Unix 
Consultant for Snow B.V. BSD Magazine asked him some 
questions regarding event organization and opportunities 
to participate in organizing EuroBSDcon. 
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Nmap 


The Network Swiss Army Knife 


Nmap (“Network Mapper”) is a GPL utility for network 
discovery and security auditing. Many systems and network 
administrators find it very useful for network inventory, 
monitoring hosts and services uptime, debugging network 
related problems, and many other tasks. 


What you will learn... 
« the basic functionalities of Nmap 6 


released. This release comes with some new fea- 
tures and many improvements. 
To install nmap on OpenBSD just run the command 


A fter three years of development Nmap 6 has been 


pkg add -i nmap. 

lf you want to install the gui as well: pxg add -i nmap- 
zenmap. 

In OpenBSD nmap has been recently updated to the lat- 
est version: 6.01 If you want to test all new 

improvements you should install a snapshot or wait for 
the 5.2 release of OpenBSD. 

Nmap is mostly used to check known hosts for open, 
closed or filtered ports. To do this execute it with just the 
name of the host you want to scan: Listing 1. 

lf you want to know more info about your target just add 
some options: Listing 2. 

By adding “-A” option you ask nmap to let you know 
more informations about the target you are scanning; the 
“-T4" option increases nmap's speed of execution (one of 
the improvements of nmap6). Keep in mind that the faster 
nmap is scanning, the easier it will be for someone to no- 
tice, either by seeing the kind of the packets nmap gen- 
erates as they're travelling on the wire or by noticing de- 
graded performance on the system being scanned. 

In Nmap 6.00 there are many more nse scripts than in 
previous versions; the Nmap Scripting Engine (NSE) is 
one of Nmap's most powerful and flexible features. 
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What you should know... 
¢ basic tcp/ip knowledge 


It allows users to write simple Lua scripts to automate 
network tasks including vulnerability detection and exploi- 
tation. For example, to test whether our web server has 
any known vulnerabilities we can run: Listing 3. 


Listing 1. Localhost scan 

Sy igiuicyey lead OO I 

Crating Nmaps oO. Ol Minos /mmap oc.) ec Ul7 =O — 77 
Da oC OL 

Nmap scam repore. for localhosc (127 .0.0 21) 

Host as up (00000545 latency). 


Not shown: 993 closed ports 


PORT STATE SERVICE 
13/tcp open daytime 
25/ €ep, | Coen smtp 
37/tcp open time 
113/tcp open ident 
587/tcp open submission 
631/tcp open ipp 


6000/tcp open X11 


Nmap done: 1 IP address (1 host up) scanned in 17.34 


seconds 
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Listing 2. Service detection scan on localhost 


Simao =k) P44 127.050 


Searcimg litapeo. Ol shibton, (mine .ong esate” OZ O77 a2 3 Oi enon 

Nap sean wepore for localnoseu, (2/7. 020, 1) 

Host ws up (00000585 lavency): 

Noe shown: [99S closed pores 

PORT STATE SERVICE VERSION 

Z25/ tcp open smtp Sendmail 8.14.5/8.14.5 

| smtp-commands: bigio.snb.it Hello giovanni@localhost [127.0.0.1], pleased to meet you, ENHANCEDSTATUSCODES, 
PIPING -sBliMiIME Si7n,) Don, Bin, DEI VERE) Hine, 


pez OOS tiirs Ts ssendiianh version sl 4 so 2.030 loptes: 2.050 EO EhhOOMA RCE PALA 200k Sei NOOP OUT HE iin Vink 


20.0 °E XEN VERB ORERN DSN AUTH 22050 STARTINS 2.000 for more Intro wse VHRMP <eopie> 2 2.0.0) To 
report bugs in the implementation see 2.0.0 http://www.sendmail.org/email-addresses.html 2.0.0 
For local intormatron send email to Postmaster at your site. 2.0.0 End of HELP into 
631/tcp open ipp Cures 1.5 
http-methods: Potentially risky methods: PUT 
_See http://nmap.org/nsedoc/scripts/http-methods.html 
NEED=KObOrS.Exu: Iidisallowed entry 
a 
6000/tcp open X11 (access denied) 


Service Info: Host: scan test. lane OS: Unix 


Service detection performed. Please report any incorrect results at http://nmap.org/submit/ 


Nmap done: 1 IP address (1 host up) scanned in 24.57 seconds 


Listing 3. Nse scripts check onan http server 


S mapea=se. pol. U2 70 0rd 


Nmap escan vepore “for Localnose (17) 20,021) 

Host 2s Up (0. 000G3s latency): 

PORE STATE SERVICE 

80/tcp open http 

[Rnbtp title. teste Page for Apache Wuctaltarien 

| http methods: No Allow or Public header 1m OPTIONS <esponse (Status code 405) 


Nmap done: 1 IP address (1 host up) scanned in 0.97 seconds 


Listing 4. Ipv6 scan 
ST inieiOa = 4 Sy aml 


Stactime Nmap o.0l i htt, /mmep ,org at 207-072 7 23 AS Gret 
Nmap Sean report for localhost 2:1) 

Host is up (0.00225 latbency )- 

POR STATE SERVICE 

25/tcp open smtp 


Nmap done: 1 IP address (1 host up) scanned in 0.28 seconds 
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Listing 5. nping testing on localhost 


S sicko, moa oS enie. Mee 108 10) 1 
Starting Noung 0.6.0 ( http.) mao ord, mong } 
SENET (0200675) 
RCVD (0200758) 
O@e2 ims Oea27 ims 

(0700s) 


Max EEG: PSM int 0e 52 ims 


(232) >) |" Revd i 


| Avg Pre: 


Raw packets sent: 1 (C2 | iersie = 10 


ae 2 ONO ee eho 
ICMP 127.0.0.1 > 127.0.0.1 Echo request (type=8/code=0) 
ICMP 127.0.0.1 > 127.0.0.1 Echo reply (type=0/code=0) 


i tame sO MOO Leen) Wx bytes(ce ole 50) Pie yekic/ eo. y ole 27 
Re tame. I O0729s || Rx bytes/s: 927.380 || Rx pkts/s: 0.99 
Nping done: 1 IP address pinged in 1.02 seconds 


ttl=64 1d=45674 iplen=28 
ttl=255 id=47169 iplen=28 


One of the main improvements to Nmap 6.00 is full ipv6 
support; all options now 

support ipv6 addresses. Just add the “-6” option to your 
command line: Listing 4. 

Some nmap options need root access (for example “-O” 
parameter used to detect the remote operating system 
version), but most options works when nmap runs as an 
unprivileged user as well. 


Nping and Neat 
In recent Nmap versions (5.00 and later), a couple of new 
tools have been added: “nping” and “ncat”. 

Nping is a tool for network packet generation tool capa- 
ble using a wide variety of protocols. It can be used for raw 
packet generation, network response analysis, network 
stack stress tests, route tracing, and more (Listing 5). 


Saat Tools Fofle Bele 


Target: | 19 LL = Freig 


Comma: nicer 0 Ae PL ee 9 
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BH lecaihest (127 
er ee 107.1 


Abcred porta: 4943 
fia pete a 
Roe pa 
Up tem 
Last boot Pr dug 2105S Hs 
[hdres 

Bel: De LoS 

Be: et cvdelabhe 

Mar. 2 ae 
r 7 
Sani e Bicrasait Wario. F Prot iepsonal 
Aare ; 
© Pore uepedl 
2 OS Clannes 
STP Sanaa 
© DP OG Sepquctrtte: 
TCE TS. tence 


— Corr rr cervte. 


iE-or Hoyts 


LEE Pa eee) Hee De: 


Figure 1. Service detection scan using nmap gui 
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Ncat is a feature-packed networking utility which reads 
and writes data across networks from the command line. 
Ncat is designed to be a reliable back-end tool to instantly 
provide network connectivity to other applications and us- 
ers. 

It is frequently used to create simple proxies for other 
applications. 

For example, to create a simple http proxy server just 
type the following command line: 


ncoat -l —-proxy-type http Localhost 8080 


Zenmap: Nmap for Everybody 

nmap also has gui interface named Zenmap; with Ze- 
nmap you have all of nmap’s options available; you can 
scan hosts, networks and have all fancy reports you want 
with just few clicks. 

As a plus you can save your scans in an xml config file 
to repeat it later. 

With the new nmap you can probe for open, closed, fil- 
tered ports on remote hosts and discover which operating 
systems remote hosts are running even faster than in pre- 
vious releases; you can also save your scanning results in 
many file formats which can be used for post-processing 
with other tools. 


GIOVANNI BECHIS 

Giovanni Bechis lives in Italy with his wife and son. He is an 
OpenBSD developer and the owner of SnB, a software house 
which provides web and hosting solutions based mainly on *BSD 
systems. He can be reached at http://www.snb. it. 
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What's New 


in PC-BSD 9.1 


PC-BSD 9.1 is expected to be released during September, 2012. This 
article introduces some of the new features of this release. 


more graphical utilities available within Control 

Panel, a redesigned installer, a server installation 
wizard, and improved jail management. This article intro- 
duces these new features. 


> C-BSD 9.1 adds many new features, ranging from 


New Control Panel Utilities 
Control Panel was introduced in PC-BSD 9.0, providing 
common access to graphical configuration utilities, re- 
gardless of the desktop one is logged into. 

PC-BSD 9.1 adds several more graphical configuration 
utilities: 


1. The About icon, seen in Figure 1, makes it easy to de- 
termine the PC-BSD version, the hostname of the 
system, the versions of the desktops which are in- 
stalled, and the version of X that is installed. 

2. The Active Directory & LDAP utility, seen in Figure 2, 
allows you to set the client information for connecting 
to Active Directory or LDAP servers. 

3. The EasyPBI utility started out as a FreeBSD port and 
is used to automate the conversion of a FreeBSD 
port into a PC-BSD PBI. It is now available through 
Control Panel. The improved design supports ad- 
vanced options such as configuring additional ports 
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PC3SD 


to build before or after the PBI build, modifying the 
desktop and menu entries, and adding post-installa- 
tion scripts. Once the PBI module is complete, it will 
package the module so that it can be submitted to 
the PC-BSD PBI build server. A screenshot is seen 
in Figure 3. 


About wis) 
Version: $.1-BETA1 (amdé4) 
Host: pcbhsd-7456 
( Back ) 
X.org server version: 1.10.6 
Desktop environments: 
Name Version 
KDE 4.8.4 1 
Close 
Figure 1. About Utility 
09/2012 
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4. The GDM Configuration utility, seen in Figure 4, can 
be used to configure a user account for automatic 
login. It can also be used to configure remote login 
through XDMCP. 

5. The Hardware Compatibility utility, seen in Figure 5, is 
available both during installation and afterwards us- 
ing Control Panel. It provides a quick overview of de- 
tected hardware devices and indicates whether or not 
the system's video, Ethernet, wireless, and sound de- 
vices are compatible with PC-BSD. 

6. The Mount Tray utility is available in both Control 
Panel and the System Tray. It allows easy access to 
mounted partitions and USB drives. If you insert a 
USB drive, a pop-up message will indicate that a new 
device is available. If you right-click the Mount Tray, 
as seen in the example in Figure 6, you can choose 
to mount or automount the device. You can also ac- 
cess the mounted partitions using the desktop's de- 
fault file manager by clicking “Open Media Directory”. 


AD and LDAP Configuration we) ty 


LDAP 
Enable Active Directory 


Active Directory 


Figure 2. Active Directory & LDAP Utility 
| File About 


New Module 


Part Selected nettrickle Get Port Info 


Program Mane | trickle 

Program Website | httpe//monkeyorg/=marius/trickle/ 
Part Author gahr@rreeBsSD.org 

Menu Category | Network 

loon fhome/dru/EasyPBldefaulticon.png Choose icon 


Create Desktop/Menu Entries 


Create Module © Aeset form 


Please fill out the above fields and click "Create Module’, 


: 
, 
} 
2 
a 
é 
3 
3 
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Figure 3. EasyPBl Utility 
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7. The Sound Configuration utility, seen in Figure 7, can 
be used to test sound or change the default audio de- 
vice. The drop-down menu can be used to determine 
which audio devices are available and to change the 
default device. A “Test sound” button is provided to 
test the selected audio device. 


Redesigned Installer 

The PC-BSD 9.1 installer has been redesigned to allow for 
OEM installs as it separates installation tasks from post- 
installation configuration tasks. Installation tasks include 
determining which system components to install and the 
disk layout to use. Post-installation configuration tasks in- 
clude setting the timezone, the administrative password, 
and creating the initial login account. The redesign also 
simplifies the installation process. A default installation 


Auto login | Remote login | 


Enable auto login 
Auto login user 


Delay 


a? 


Figure 4. GDM Configuration Utility 


= 


. Hardware Support 


Detected hardware devices 
Wl Video driver: (nvidia) 

bes) Video resolution: (1600x900) 
eS Ethernet device: (em0) 


Wifi device: (iwnQ) 


Se No sound detected 
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can begin after 4 mouse clicks. In PC-BSD 9.1, a default 
installation is defined as follows: 


e installation occurs on the entire primary drive 

¢ if the system has less than 2GB of RAM, that drive 
is formatted with UFS. Otherwise, it is formatted with 
ZES} 

¢ if the system has less than 2GB of RAM, the LXDE 
desktop will be installed. Otherwise, the KDE desktop 
will be installed. 


For users who wish to change the default installation 
partition, filesystem, or desktop, each installation screen 
contains a Customize button. Figure 8 shows the options 
which are available when the Customize button is select- 
ed in the Desktop Selection screen. 

The Customize button of the Disk Selection screen of 
the installer now supports three modes of operation: 


¢ Basic: (default) used to specify which partition or disk 
to install to and to configure encryption. 

¢ Advanced: used to specify the installation partition or 
disk, GPT partitioning, encrypt user data, disable the 
FreeBSD boot menu, or specify the filesystem to use 
and the layout of that filesystem. 

¢ FreeBSD Expert: used to drop down to a shell to 
manually enter the commands to configure the disk 
layout. 


ZFS configuration has been improved. If you wish to 
add multiple drives to the ZFS pool, the installer will indi- 
cate the minimum number of drives needed for a mirror, 
RAIDZ1, RAIDZ2, or RAIDZ3. The installer also allows 
you to select the following ZFS properties for each ZFS 


USB DISK Mount da0s1 


Auto-mount 


™ Open Media Directory 


E43 Close Tray 


Sound Configuration 
Below you may change the default sound device, and test sound playback 
pem2: <Conexant CX20590 HDA CODEC PCM (Analog)> (play/rec) default = 


Test sound 


Close 


Figure 7. Sound Configuration Utility 
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mount point (dataset): atime, canmount, checksum, com- 
pression, and exec. 


Server Install Wizard 
The redesigned installer also adds a server install wizard 
capable of installing two types of servers: 


¢ FreeBSD Server: installs a basic, vanilla installation of 
FreeBSD. While the installation routine is different, the 
end result is the same as if one had installed FreeBSD 
from a FreeBSD media as it results in a minimal, com- 
mand line only FreeBSD server installation. 

¢ TrueOS™: adds the following features to a vanilla in- 
stallation of FreeBSD: the PB] Manager command line 
suite of utilities which can be used to manage PBIs 
and create one's own software repositories, a com- 
mand line utility for managing system components, 
a command line utility for managing updates, and 
the command line version of Warden® for jail mana- 
gement. 


Besides providing a graphical installer, using PC-BSD to 
install a server offers the following advantages: 


¢ the ability to easily configure ZFS during installation. 

e the ability to configure encryption during installation. 

¢ a wizard to configure the server for first use. This wiz- 
ard is used to configure the system host name, root 
password, primary login account, enable SSH, con- 
figure networking, and install src or ports. 


Improved Jail Management 

Warden®, PC-BSD's utility to manage jails, has been 
completely redesigned for 9.1. It no longer needs to be in- 
Stalled as it is part of the base system and available from 
Control Panel. Some of its new features include the abil- 
ity to: 


Figure 8. Customizing the Desktop 
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¢ create three types of jails: a traditional FreeBSD jail 
for running network services, a (less secure) ports jail 
for safely installing and running FreeBSD ports/pack- 
ages from a PC-BSD system, and a Linux jail for in- 
stalling Linux (currently installation scripts are provid- 
ed for Gentoo and Debian) 

¢ set multiple IPv4 and IPv6 addresses per jail 

¢ quickly install meta-packages of common network 
server applications on a per-jail basis 

¢ use Update Manager to manage software and system 
upgrades on a per-jail basis 

¢ use User Manager to manage user accounts on a 
per-jail basis 

¢ manage ZFS snapshots on a per-jail basis if the PC- 
BSD system is formatted with the ZFS filesystem 

¢ export a jail which can be then be imported into the 
same or a different jail 


Warden® provides a graphical interface for the PC-BSD 
desktop and a command line version for a TrueOS™ in- 
Stallation. Figure 9 shows an example of a system with 
three jails installed, one of each type. 


The main screen of Warden® provides an overview of 
each jail as well as buttons for stopping and starting the 
highlighted jail. 

The tools tab provides the following buttons: 


¢ User Administrator: opens User Manager to manage 
the highlighted jail's user accounts and groups. This 
button is not available if a Linux jail is highlighted. 

¢ Service Manager: opens Service Manager to view 
which services are running in the jail and to config- 
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Figure 9. Warden® Graphical Interace 
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ure which services should start when the jail is started. 
This button is not available if a Linux jail is highlighted. 

¢ Launch Terminal: opens a terminal with the root user 
logged into the jail. This allows you to adminster the 
jail from the command line. 

¢ Check for Updates: launches Update Manager to de- 
termine if any of the jail's installed applications have 
newer versions available. Update Manager will al- 
so indicate if system updates are available to be in- 
Stalled into the jail. This button is not available if a 
Linux jail is highlighted. 

¢ Export Jail: used to save a backup of the jail and all 
of its software, configuration, and files. 


lf the PC-BSD system was formatted with ZFS, the 
Snapshots tab can be used to manage snapshots, or 
point-in-time copies of the filesystem. Since jails share 
the filesystem used by PC-BSD, any type of jail, includ- 
ing a Linux jail, can take advantage of this ZFS fea- 
ture. This tab provides buttons to create, delete, restore, 
mount, and unmount snapshots. 

The packages tab allows you to install meta-package 
software which will be tracked by Update Manager for 
newer versions. Common server applications are avail- 
able, such as databases, web servers, file servers, and 
programming languages. 


Summary 

PC-BSD 9.1 introduces many new features which are de- 
signed to make it easier than ever to install and configure 
a desktop or server based on FreeBSD. You can learn 
more about how to use these features in the PC-BSD 
9.1 Users Handbook which is provided as an icon on the 
desktop of an installed release. You can read a preview of 
this Handbook prior to release at the PC-BSD documen- 
tation wiki: htto://wiki.pcbsd.org/index.php/PC-BSD_Us- 
ers_ Handbook. 
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OwnCloud Instance via The Warden™ 


With the increase of smart-phones, tablets and the general mobility 
of users, we are also seeing an increase in the interaction with 
applications and data online - commonly known as the “Cloud”. 


ith this change in behavior, users are becom- 
VAY A ing increasingly aware of the potential privacy 

and security issues that are associated with per- 
sonal data being stored offsite. Companies, governments, 
or even nefarious individuals could easily obtain access to 
this data for whatever purposes they so deem. With this 
ongoing trend, we have begun to see new software be- 
come available which allows users to host their own private 
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Figure 1. Starting the Warden GUI 
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“Cloud” at whatever location they wish, even from their own 
home desktop or server. In this article we will be taking a 
look at the OwnCloud software, specifically how to do the 
initial installation and configuration inside a jail run by PC- 
BSD's® jail management utility, the Warden™. First we will 
take a look at a setup done from a PC-BSD graphical inter- 
face, and then explore the same setup from the command- 
line using TrueOS™, the server version of PC-BSD. 


New jail Wizard 


| 
| er 


| ‘| This wizard will walk you through creating a new jail, First, 
enter the new IP address and hostname and click next to 
continue. 


IP Address 
192.168.0.45 
Hostname 
owncloud)ail 


-GBeck )| Next> | Cancel 


Figure 2. Assigning an IP address and hostname 
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Setting up Your OwnCloud Instance via The Warden™ 


Creating the jail via the Warden GUI 

To begin, you will need to start the Warden GUI via the 
Control Panel (Figure 1) Next, you will want to ensure that 
the jail is running on the correct network interface in Jails 
— Configuration. Via the pull-down menu, select the net- 
work interface you want to run your jails on. Normally the 
selected interface should be the same interface you are 
using to connect to your network and the Internet. 

Once the jail interface has been set properly, go to 
File — New Jail to start the creation process. On the first 
screen, you will need to assign an IP address and host- 
name to this new jail, and then click “Next” to continue 
(Figure 2). 


Note 

The IP address should be a unique address on your net- 
work, not the same as your host's IP. For example, if your 
system's IP address is 192.168.0.100, then you could pick 


New jail Wizard -oOx 
.— 
| 12 Please select the type of jail you want to create, 
Jall Type 
* Traditional /ail (Secure, best for server applications) 
Parts jail (Insecure, allows running A applications) 
Linux jail (Run Linux within iin & jail) 
“Back | Next> |) Cancel 
Figure 3. Jail type selection 
New jail Wizard - 8 
12 Please take A moment and set any other options for this 
= jall. Note that Including the system source and ports tree 
options require that they be present in /usr/src and 
fusr/ports respectively. 
Jail Options 
Include system source 
Include ports tree 
‘| Start jail at system bootup 
«= Back | Finish |) Cancel 


Figure 4. Setting the jail options 
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something unused in the 192.168.0.xxx range, or another 
address on the same subnet as the host. Next, select the 
type of jail you are creating. In this instance, you will be 
using a “Traditional Jail’, select it and click “Next” to con- 
tinue (Figure 3). On the next screen, you will need to enter 
a root password for this jail and click “Next” to continue. 

Lastly you can set additional options for this jail. If you 
plan on building software from FreeBSD ports, you can se- 
lect to install the ports tree, and system sources. If you want 
this jail to run every time the system boots, you may wish 
to also check “Start jail at system bootup”. When you are 
ready, click “Finish” to begin the jail creation (Figure 4). This 
may take a few minutes the first time you create a jail, be- 
cause a fresh jail environment needs to be downloaded. 

After the jail creation has finished, you will then need to 
install the software required to run OwnCloud. OwnCloud 
is written in PHP, and requires access to a database, such 
as MySQL. In addition you will need a web-server, such 
as Apache, to serve the site. 

Using the Warden GUI, you can select your new jail and 
click “Packages” to browse for and select the packages 
MySQL, PHP and Apache to the jail. Click “Apply” to begin 
installing them (Figure 5). Once the packages have finished 
installing, start the Apache and MySQL services inside the 
jail. You can do so on the “Tools” tab of the jail manager by 
selecting the “Service Manager” button (Figure 6). 
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Figure 5. Selecting the server packages required for OwnCloud 
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Figure 6. The available tools for a FreeBSD traditional jail 
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In the Services Manager, you will be given a list of 
services for this jail. Scroll through the list, select the 
“apache22” service, and click “Enable” and then “Start” 
(Figure 7). Repeat the process for the mysql service. 

lf everything has been successful to this point, you should 
be able to bring up your web-browser, point it to the jail's IP 
address, and see the text “It works!” Congratulations, you 
are now ready to setup your OwnCloud software. 


Note 

lf you are trying to connect from a web-browser on anoth- 
er system, you may need to open port 80 in the Control 
Panel — Firewall utility first). You may skip the next sec- 
tion and jump down to Configuring the jail for OwnCloud. 


Creating the jail via the Warden Command-Line 
Users who do not wish to run a full desktop operating 
system may still use the Warden via a command-line in- 
terface after installing TrueOS (which is included on the 
PC-BSD install DVD). To begin, you will need to configure 
it to use the correct network interface for your jails. This 
is done by editing the file /usr/local/etc/warden.conf, and 
changing the interface line as shown below: 


NIC: reQ0 


With this configured, you are now ready to create the 
new jail. Use the following command, changing the host- 
name / IP address to your preference. 


# warden create 192.168.0.45 owncloudjail --startauto 


With the jail now created, you will need to install the 
packages required for running an OwnCloud Server. Us- 
Ing the built-in pc-metapkgmanager command, YOu Can do 
so with the following command: 


Managing services for Warden IP: 1927.168.0.45 


Service Name =) Running Enabled 
accounting Disabled 


amd Cisabled 


Disabled | 
Disabled 
Disabled 
Disabled 
Cisabled 
Suditd Disabled 
background-fsck Stopped Disabled 
bluetooth Disabled 
bootparamd Disabled 
BSsrmpad Disabled 
bthidd Cisabled 
cleanvar Enabled = 
i Stop <x Restart Disable Service 


Figure 7. Enabling the services for this jail 
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# pc-metapkgmanager --pkgset warden --chroot 


/usr/jails/192.168.0.45 add MySQL, Apache, PHP 


Once the packages have finished installing, you will 
need to enable them to run at startup by editing /etc/ 
rc.conf from within the jail. This can be done using the 
following commands and by adding the lines apache22 _ 


enable="YES” and mysql enable="YRS@itow cue, menconn, 


# warden chroot 192.168.0.45 
root@owncloudjail:/ # vi /etc/rc.conf 
root@owncloudjail:/ # /usr/local/etc/re.d/apache22 start 


root@owncloudjail:/ # /usr/local/etc/rce.d/mysql-server start 


Congratulations, you are now ready to setup your Own- 
Cloud software. 


Note 

lf you are trying to connect from a web-browser on anoth- 
er system, you may need to open port 80 in the Control 
Panel — Firewall utility first). If running from TrueOS™ 
you may need to add an exception into /etc/pf. cont. 


Configuring the Jail for OwnCloud 

To install the OwnCloud software, fetch it and extract it 
into the jail via the shell prompt. To open a shell, navi- 
gate back to the “Tools” tab of the jail and click “Launch 
Terminal’, or from the command prompt run warden chroot 
192.168.0.45 replacing with your IP. Once the shell or termi- 
nal has started, type the following commands to download 
and extract your OwnCloud. 
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Figure 8. OwnCloud Setup Screen 
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# cd /usr/local/www/apache22/data 

# fetch http://download.owncloud.org/releases/owncloud- 
me O.6.tar.bz2 

i tar xgemermmelome.4.0.6.tar.bz2 


# chown -R www:www owncloud 


With this done, you will now have a new /usr/local/www/ 
apache22/data/owncloud directory, waiting to be setup. Be- 
fore you close the shell, create a new MySQL data- 
base, and configure PHP properly for your OwnCloud in- 
stance. To create the database, use the commands be- 
low, changing the password to one of your liking. 


# myseiee—u root 

mysgl> Create database owncloud; 

itjmee'> Grant all on owncloud.* to ocuser@localhost 
identified by "mypass"; 

mvyeql> quit 


After configuring MySQL, you need to enable some ad- 
ditional Apache PHP options. Open the file /usxr/10ca1/ 
etc/apache22/httpd.conf, USINg your favorite editor, such as 
“vi’ or “edit”, and browse for the following section: 


# AddType allows you to add to or override the MIME configuration 
# file specified in TypesConfig for specific file types. 

if 

#AddType application/x-gzip .tgz 

if 


Add the following lines right below this section: 


# AddType allows you to add to or override the MIME configuration 
# file specified in TypesConfig for specific file types. 

if 

#AddType application/x-gzip .tgz 


# Files 


Ji Music “ 
Calendar 
4% Contacts 


M@ Pictures 


Figure 9. OwnCloud main screen 
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# 
AddType application/x-httpd-php .php 
AddType application/x-httpd-php-source .phps 


Lastly, search for the following section: 


<1 iModule=dicinccme. 
DirectoryIndex index.html 


</1TiModule> 
and add: 


<I fModule dur sncemie, 
DirectoryIndex index.html index.php 
</IfModule> 


With these changes made, save the httpd.conf file, then 
restart Apache with the following command: 


# /usr/local/etc/rc.d/apache22 restart 


With this configuration done, you are now ready to 
launch OwnCloud. In your browser, navigate to the URL: 
http://192.168.0.45/owncloud/, replacing “192.168.0.45” 
with your jail IP. When you bring up the page, you will be 
presented with a first-time setup screen. Create a new 
user and password, and be sure to click the advanced 
button. In the advanced settings you will need to enter 
the MySQL username, password, and database name 
you previously created (Figure 8). Click “Finish” to fi- 
nalize the OwnCloud configuration, and enter your new 
cloud interface! 

With OwnCloud setup and configured properly, you 
should be taken to the main interface screen (Figure 9). 
From here you can now begin to use it to store files (ala 
DropBox), manage your calendars, contacts, and much 
more. 

By clicking the small “gear” icon in the bottom left, you 
can further customize your Cloud account, locate the Cal- 
Dav, CardDav and WebDav addresses for mobile devic- 
es, install 3rd party applications and more. For more infor- 
mation on using the OwnCloud interface and integrating 
with your mobile device, you may wish to read through the 
documentation and guides located on the OwnCloud sup- 
port site. (http://owncloud.org/support/. 
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Unix IPC with Pipes 


This article explains one of the earliest forms of inter-process 
communication (IPC) in Unix. Pipes were the original form 

of Unix IPC and were present in Third Edition of Unix (1973). 
They can only be used to communicate between related 
processes, but despite this limitation they still remain one of 
the most frequently employed mechanisms for IPC. 


What you will learn... 
¢- How pipes are created 

¢ How processes use pipes 

« File descriptors 

- The ‘fstat(1) command 


he ¢stat(1) Command, which first appeared in 

| 4.3BSD, displays the status of open files, sockets 

and pipes (as well as other objects) on a system 

and provides information on their I/O activity. To under- 

stand the output of stat (1) it is necessary to know what a 

‘descriptor’ is and how it is used to identify an access path 

for I/O from a userland program to a disk, network socket, 
pipe, etc. 


Descriptors 

Descriptors are used within programs to reference ‘ob- 
jects' used for I/O. Typically, these objects refer to files, 
pipes or sockets; less common are event queues for no- 
tification of kernel events, and the ‘crypto’ object which is 
used for direct access to cryptographic hardware. Addi- 
tional types exist, but their definition and presence varies 
from one BSD to the other. 

The descriptor is an integer which is allocated by the 
kernel when a program executes the appropriate system 
call to open the the object: pipe.) for opening a pipe, open () 
for opening a file, or socket () for local or network sockets, 
etc. All system calls which perform I/O on the given object 
or modify its parameters will reference the object using 
the descriptor. A descriptor remains allocated (‘open’) until 
it is either closed by the process or the process exits. 

Most applications, including shells, associate file de- 
scriptors 0,1,2 with standard input, standard output, and 
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What you should know... 


¢ Basic command line operations 


standard error, respectively. There is a limit on how many 
descriptors a process may have open at any given time. 
This is defined by the OPEN MAX constant, and ranges 
from 64 (FreeBSD) to 128 (NetBSD) and on OpenBSD, 
from soft limit of 64 to a hard limit of 1024. 

Associated with every process is a table of open de- 
scriptors, and each descriptor is merely an index into this 
table. The descriptor table holds a reference to a file en- 
try. The kernel maintains a table of file entries for all open 
objects in the system. The file entry itself is an instance of 
the file structure. A field in the file structure identifies the 
type of underlying object — socket or pipe for sockets or 
pipes respectively, type v-node for files in the file system, 
which may include FIFOs and devices in /dev/, etc. (The 
possible values for this field vary among BSDs; it is nec- 
essary look in sys/file.h on the particular system to see 
how they are defined.) 

The file structure also holds the status flags for the ob- 
ject (e.g., read only, read-write, append, etc) specified 
when the object was opened, the current offset within the 
file where the next read or write will occur (if the object ref- 
erenced is a file), the amount of data transferred and the 
number of transfers, and the particular I/O routines spe- 
cific to that type of object. 

It is important to note that when a process Calls fork (2), 
the open descriptors in the parent process are copied 
to the child, and after the forx() the parent and child will 


09/2012 


share the same descriptors for reading and writing data. 
This will also be the case if the child calls exec(3) to ex- 
ecute a program different from that of the parent, though 
this behavior can be changed by setting the ‘close-on-ex- 
ec’ flag which is associated with a descriptor. If the flag 
is set, open descriptors inherited from the parent will be 
closed when calling exec(). 

After a fork(), both the descriptors in the parent and the 
child will reference the same file entry. This means that 
reads or writes by either process will advance the offset 
where the other process will perform its next read or write. 
Also, the values for I/O activity will be incremented by ei- 
ther process. The fstat(1) program is a tool for reading the 
table of open descriptors for a given process and return- 
ing statistics on their I/O. 

The example in Listing 1 uses fstat(1) on OpenBSD. In 
another terminal, the pager program /usr/bin/iess IS run- 
ning and its PID is passed to fstat as an argument. The 
options are: 


s — report file I/O statistics — the number of transfers and 
number of kilobytes transfered. This option produces 
no output unless fstat is run as the super-user, or the 
UID of the process is the same as the UID of the us- 
er running fstat. 

o — report file offset. This is the byte offset from the be- 
ginning of the file where the process is either reading 
or writing. 

p — the pid of the process 


The column headings in the output are: 


USER — the owner of the process 

CMD — the command 

PID — the process ID 

FD — the file descriptor number, or one of the follow- 
ing special names: 
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text — executable text inode 

wd — current working directory 

root — root inode 

tr — kernel trace file (the output file if ktrace is run- 

ning) 
MOUNT — mount point for file system where the par- 
ticular file resides 
INUM — inode number for the particular file 
MODE - file type and permissions on the file 
R/W — whether file is open for reading and/or writing 
SZ/DV:OFFSET -— if a regular file, this will be the size 
of the file followed by the current offset into the file 
where the next read or write will occur; if a charac- 
ter or block special file, the name of the device file in / 
dev 
XFERS — the number of times data has been trans- 
ferred in either direction. 
KBY TES — number of kilobytes transferred. 


N.B. Due to a small bug OpenBSD 5.1, the value for 
KBYTES are incorrect. This will be fixed in the new 5.2 
release due in November. Until the release of 5.2, the 
two required patches can be downloaded from www.tet- 
rardus.net/bsdmag/diffs/ 

The first two lines of Figure 1 show information about 
the binary executable (/usr/pin/iess) and its working di- 
rectory. Most of the information in these two lines can be 
obtained using ‘Is -li’ ON /usr/pin/less or the current work- 
ing directory. 

The following 3 lines show information for file descrip- 
tors (FD) 0, 1, 2, which correspond to standard input, stan- 
dard out and standard error. These 3 descriptors map to 
the same inode, 964 (INUM), which refers to a file of type 
character (‘c' in MODE column) and is the device file /aev/ 
ttyp9 associated with the terminal where ‘less’ is running. 
Since ‘less' was created via the fork() and exec() method 
initiated by the shell running in the other terminal and con- 


Listing 1. Output of /usr/bin/fstat 


# fstat -sop ‘pgrep less’ 


USER CMD Pip FD MOUNT INUM MODE R/W 
paul less 8348 text /usr SVS See =e 
paul less 8348 wd /home 987392 drwxr-xr-x 
paul less 8348 ers S64 en = ee a 
paul less 8348 ey, ol Cia sia= == ey 
paul less 8348 Zoe VCs rw Waa eT 
paul less 8348 oy IGG Clavie Sia 
paul less 8348 4 /var Ze a ee 


52 DV SOPRSEL EER KEYES 
b33¢4:0 0 0 

Sula 0 0 
Cie Ae) 862 1s) 
ttyp9 363 ig 
ttyp9 363 iL) 

iia 0 0 
Zo O'Se Oleg Z Z 8 
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sequently inherited the shell's descriptors that were at- 
tached to the device ttyp9, both processes now share the 
same file entry for this device. Therefore the values under 
XFERS and KBYTES reflect not only I/O activity initiated 
by 'less' but also the I/O activity generated by the shell, 
including activity that occurred before ‘less' was started. 

The next line shows that the program ‘less' has opened 
/dev/tty (INUM = 1616) for reading (R/W = r) on descriptor 
3. It is through this descriptor that 'less' reads input from 
the keyboard. Thus far, there has been no input from the 
keyboard (XFERS = 0, KBYTES= 0). 

The last line shows that 'less' has a file open on descrip- 
tor 4. The file's inode number is 12, the file is open for 
reading, the file is 25708 bytes in size, and the file's cur- 
rent offset is 8192 bytes into the file (SZ|DV: OFFSET = 
25708:8192). 8kb has been read (KBYTES = 8) and this 
required 2 data transfers. 

Again, much of this information is static (e.g., file's inode, 
size, etc) and can be obtained using options to /vin/1s. 


Pipes 

Pipes provide a fast, reliable, stream oriented method of 
uni-directional data flow between related processes. In this 
case, related specifically means processes having a parent- 
child relationship or processes having a sibling relationship 
(i.e processes that have a common ancestor) (Figure 1). 


Proces § 


Step 1: 

Process calls piped) 
fd open for writing 
AS open for nading 


Parent Process 


Step ?: 
Process calls fork) 
Parent and child share 
desc nptors fio, fa 


Child Precess 


Parent Process Child Process Step 3: 
Parent closes fd 


Child cloges fd 


Figure 1. Creating a pipe 
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Pipe creation takes advantage of the fact that, after a 
call to fork(), Open descriptors in a parent process will 
be inherited by the child. Figure 1 shows the three step 
process that creates a pipe. First, a process invokes the 
pipe(2) system call, which creates a buffer in the kernel 
and returns 2 file descriptors which reference the pipe. 
The descriptor sao is open for reading, and fai is open for 
writing. Data written to descriptor sa1 can be read on faa 
(the arrows indicate the direction of data flow). 

In step 2, the process calls forx(), and the child inherits 
the parent's open descriptors. After the fork, both the par- 
ent and the child can read or write to the pipe. 

In the final step, the parent then closes the descriptor 
which is open for reading (£a0) and the child closes the 
descriptor which is open for writing (sai). The result is a 
uni-directional data path between fd1 in the parent to «ao 
in the child. If the child were to exec() another program, the 
pipe would still be open, and the new process would read 
its standard input from the pipe. 

A pipe is a buffer in kernel memory which defaults 16kb 
in size. This buffer exists until both descriptors are closed. 
Data written into the pipe is stored in this buffer until it is 
read by the process on the other end. If the processes on 
both ends close the descriptors associated with the pipe, 
then any data left in the buffer is discarded. 

lf only one end of the pipe has been closed, the pipe is 
‘widowed’. A process writing to a pipe after the read-end 
has been closed will receive a SIGPIPE signal from the 
kernel. The default action for this signal is to terminate the 
process. A process reading from a pipe whose write-end 
has been closed will read any remaining data in the pipe 
buffer after which it will receive an EOF (end of file). The 
pipe is then in a 'end-of-file’ state and will remain in this 
state until the last descriptor is closed. 

Since the data flow between two process occurs within 
the kernel on the same host, the data transfer is reliable 
and data cannot be lost. It is also stream oriented, that is, 
the process reading data from the pipe cannot determine 
any boundaries in the data based upon writes performed 
by the other process. 

Because open descriptors are copied across a Call to 
fork(), It is possible to have multiple processes reading 
from or writing to either end of a pipe. The common case 
is one where a pipe has multiple writers and only a sin- 
gle reader. For instance, a typical configuration of Apache 
http server will spawn several children to serve client re- 
quests concurrently, and each child will write its log data 
to a single instance of a log processing program (e.g., cro- 
nolog). This is illustrated in Figure 2. 

The upper part shows the relationships after the httpd 
process has set up the pipe, but before it has spawned 
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any children. The preliminary steps are identical to what 
was shown in Figure 2: the httpd process called pipe.) 
to create the two descriptors; this was followed by a call 
fork() to create a child process. The child and parent then 
closed the appropriate descriptors to create the uni-direc- 
tional pipe. Then the child called exec) to replace itself 
with the cronolog program. 

The lower part of Figure 2 shows the relationships after 
the httpd daemon has forked 4 children. Each child has in- 
herited the open descriptor connected to the write-end of 
the pipe and cronolog is able to read the output from any 
of the httpd processes which send data to the pipe. 

An important property of pipes is that of atomicity. If 
more than one process is writing to a pipe, then there 
must be some protection against data from two separate 
processes being interleaved in the pipe buffer. To miti- 
gate this problem, the kernel guarantees the atomicity of 
writes which are less than a predefined size as set by the 
PIPE BUF constant. On OpenBSD, NetBSD, and Free- 
BSD PIPE_BUFF is set to 512 bytes. (On linux, this val- 
ue is much larger: 4096 bytes). This means that if a pro- 
cess writes data which is less than or equal to PIPE_ BUF 
bytes, then either all of the data will be written or none of 
it will. This prevents data from two processes being in- 
termixed within the pipe buffer, as one process will write 
all of its data first before the second is allowed to write 
anything. Another consequence of atomicity is that if the 
amount of data to write is less than or equal to PIPE_ BUF, 
but larger than the amount of free space in the buffer, then 
the write will not occur until there is enough space in the 
buffer to perform the write atomically. 

Most sysadmins are familiar with pipes through their 
use on the command line to redirect the standard out- 
put of one program to the standard input of another pro- 
gram (e.g., Cat <file> | grep <something>). In this case, the 
shell executing the commands uses a series of fork()$ 
and exec()S along with the pipe() system call to set up the 
pipes so that the standard output (descriptor 1) of the first 
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command writes to the pipe and the standard input (de- 
scriptor 0) of the second command reads from the pipe. 

Listing 2 is the output of fstat for exactly this scenario. 
In another terminal (not shown), the commands 'cat file__ 1 
| less' are executed and in a second terminal, fstat is in- 
voked with the PIDs of the running ‘cat’ and ‘less’ pro- 
cesses as arguments. The upper part of Listing 2 shows 
the output of ‘fstat' after the 'cat' program has read data 
from the file on disk and written it to the pipe, and ‘less' 
has read from the pipe and filled the terminal with the first 
portion of the file (The first thing to note is that the column 
headings in the output don't always apply to the lines con- 
taining descriptors which refer to pipes. Also, in this exam- 
ple the output of fstat has been piped to grep to remove 
lines which are not germane to the discussion). 

The uppermost output shows the open descriptors for 
the ‘cat’ program. Here, we're interested in descriptors 1 
and 3. Descriptor 3 is reading from a file whose inode is 
337826 and is located in a directory somewhere in the file 
system mounted on /nome. The columns SZ|DV:OFFSET 
show that the file is 5131637 bytes in length, and the cur- 
rent offset into the file is 32768 bytes (32kb). The XFERS 
and KBYTES columns show that there have been 2 data 
transfers from the file on disk which has resulted in 32kb 
having been read. This corresponds with the current off- 
set in the file, which is also 32kb. 

Descriptor 1 (FD 1), standard output, is writing to a pipe. 
Pipes are uniquely identified by a hexadecimal value. To 
the right, the values for XFERS and KBYTES are 1 and 
16. There has been 1 write operation of 16kb to the pipe. 

The second invocation of fstat is on the ‘less' program. 
File descriptor 0, standard input, is reading from the pipe. 
Here we see the value for 'state:' which is 'W', meaning a 
write operation is blocked waiting for the reader to read 
more data from the pipe. Again, the XFERS and KBYTES 
values show that the 'less' program has read 8kb of da- 
ta from the pipe. Since the device ttyp7 on descriptors 1 
and 2 is also being used by the shell, these XFERS and 
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KBYTES values reflect prior I/O activity and 
activity generated by ‘less’. 

Taken together, the output shows that ‘cat’ 
first read 32kb from the file into its internal buf- 
fers and then wrote 16kb to the pipe. The size 
of the pipe buffer is 16kb so ‘cat’ filled the pipe 
to its maximum capacity. ‘less' read only the 
first 8kb of data from the pipe, leaving 8kb in 
the pipe. We know that 'less' has written data 
to 'standard output’ on descriptor 1 (because 
we can see the contents of the file in the ter- 
minal window), but we don't know exactly how 


Figure 2. Multiple processes writing to single pipe with 1 reader 


www.bsdmag.org 


much was written because the XFERS and 


BSD . 


MAGAZINE 


HOW TO 


Listing 2. Pipe //O statistics output by fstat 
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USER CMD 
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USER CMD 
paul less 
paul less 
pau less 
oreiuul less 
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IBS NECIES = s1Oe) 1618) 


USER CMD 
pau Cal 
paul Cale 
paul Ga 
paul Cat 
Estat =sop. di 
USER CMD 
paul less 
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rep cat’ | grep -ve text -ve wd 
PID FD MOUNT INUM MODE R/W 54) DV;OFFSED XFERS 
14875 Oy 960 Cia =Wa=a= Ay Eeyod 44 
14875 1 pipe Oxfffffe80b1f37da0 state: 1 
14875 oy 960 CliWe We = == rw eyo! 44 
14875 3 /home 337826 SAW ei ta Wi ig SG sl 32766 2 
rep cat’ | grep -ve text -ve wd 
PD) FD MOUNT INUM MODE R/W 52) DV OFESE XFERS 
NS 0 pipe Oxfffffe80b1f37da0 state: W il 
11905 ye C0) Ci teyo/ 44 
11905 Ly 60 Cites == iat teyo/ 44 
11905 Se ele erw—-cw. ew r ey 0 
after paging farther into file 
rep cat’ | grep -ve text -ve wd 
EAL ID) FD MOUNT INUM MODE R/W 54) DV OFFSET XPERS 
rag S 0 / 960 Casa a= EY Eeve! 259 
14875 1 pipe Oxfffffe80b1lf37da0 state: oe) 
Las 75 2 960 Cle Sa EEyod 1259 
14875 3 /home 337826 (vw w= ie 91316372 933040 60 
rep cat’ | grep -ve text -ve wd 
PAD) He eMOUNT INUM MODE R/W 54) DY ,OERSET XFERS 
ILIKE 0 pipe Oxfffffe80b1f37da0 state: W IEG 
IDESOs ates CO emt == = ena eyo! 1259 
11905 oat Gi) SIG Cay eta i bey ib Ak 
#fstat output after paging to end of file 
| grep -ve text -ve wd 
USER CMD Pip FD MOUNT INUM MODE R/W 54) DV OFFSET AFERS 
paul less iOS OQ pipe OxEtErPesObirsidall State: 628 
pau less LOS ey: CO seri Wn Ly eiod 6671 
paul less 11905 Ly S60) Cie sia= == ey teyo! 6671 
paul less EOS Soa | GING ies iets iaiy= ‘a ey, S20 
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KBYTES values include prior I/O activity generate by the 
shell. However, since we know that ‘less' read 8kb from 
the pipe, and the value for KBYTES on descriptor 1 is only 
6, we can conclude that ‘less' didn't write all the data it had 
in its internal buffers. 

The second part of Listing 2 shows the ‘fstat' output 
for the same processes after paging substantially farther 
down into the file. 'cat' has read 960 bytes of data from 
the file on disk on descriptor 3, the new offset is at byte 
983040, and 940kb of data has been written to the pipe on 
descriptor 1. ‘less' has also read 928kb from the pipe and 
written that to the terminal. 

The lower portion of Listing 2 shows ‘fstat’ output after 
paging to the very end of the file. The ‘cat’ program, after 
writing the last amount of data to the pipe, exited and the 
pipe is now ‘widowed! (the pipe's state is 'E'). The ‘less’ 
program performed a total of 628 read operations on the 
pipe, transferring 5011kb of data. 

Although the illustration of pipes in this article has been 
limited to half-duplex communication between processes, 
it is possible, to establish full-duplex inter-process com- 
munication using two pipes, one for each direction of data 
flow. 


Summary 

Pipes are the most basic type of Unix IPC, and one of 
the most commonly used mechanisms for passing data 
between programs. They offer fast, reliable data transfer 
between related processes. Within a program, a pipe is 
referenced using a file descriptor. File descriptors identify 
instances of objects which are used for I/O in a program. 
The fstat program is a tool that reads the table of open 
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Enterprise Search with Apache Solr Part 1 


Back office integration and cross platform search has always 
posed major challenges especially in large organizations 
with many legacy systems. With Apache Solr these barriers 
can be overcome and the power of enterprise search 


realised. 


What you will learn... 
¢« How to commission an Apache Solr search engine 


lueware, middleware — call it what you want — is 
the bread and butter of the well connected en- 
terprise. Legacy systems, which may not have 


the benefit of open API's, vendor support or even an im- 
port / export facility challenge the systems integrator with 


What you should know... 


¢ BSD administration skills 


a major paradox. Often these systems are critical to the 
business, but are so culturally embedded in the busi- 
ness model that to replace them is unthinkable, either 
on the basis of functionality (The users like it) or cost 
(Too expensive to replace). Worst still, an organisation 
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can reach a dead-end in that the system is no longer 


maintainable or extensible but at the same time extra endless 
functionality is essential — e.g. exposing back office data —— 
to web. = 


Traditionally, integration is accomplished by batch oper- 
ations, import / export of data, using a messaging system 
or some form of trigger e.g. a request for specific data via 
XML. This is fine if the data set is well defined and we are 
working with “known knowns” and this model works well 
for integration as well as search. For example, searching 
for a known surname in a surname field the user searches 
for a surname “Somerville”, and will expect either a match, 
multiple matches or no result. Unless there is some other 
search technology applied the user will not quickly find 
“Sommerville”, “Somervile” or even “Summerville”. If we 
take a step back from the historical methods (Figure 1), it 
is clear that a new search paradigm is now being adopted 
by innovators on the web — faceted and intelligent (“deep”) 


search. This new search is a mixture of technologies (e.g. =“ wuEe 
Ajax, XML), data structure (Pre-defined, undefined), in; |" " "=" 


dexing (Static, dynamic) which revolutionizes the way the 
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user engages with the search process itself. No longer is 


Figure 3. Example of an innovative website with faceted search 
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Figure 2. Solr faceted search, algorithms and schema’s. The format of the binary index is dictated by Apache Lucene 
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manual keywords or tag- 
ging, but databases, and 
disparate files themselves 
can be searched for con- 
tent (Figure 2). This results 
in a major leap in func- 
tionality and the ability to 
find exactly what is want- 
ed in an ordered and logi- 
cal fashion. With the ad- 
dition of taxonomies and 
algorithms, search starts 
to become highly intelligent 
— for example related con- 
tent and facets (e.g. colour, 
price etc.) (Figure 3). 
Extending this philoso- 
phy further still — what is 
an internet search engine? 
Effectively it is the middle- 
ware between the user and 
billions of pages of dispa- 
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Download from 


apache-solr-4.0.0-BETA.tgz 


rate content on millions of web-servers. Of course. if the 
user is aware of the address of a site they can go directly 
there with the browser, but more frequently users search 
for strongly related terms and select a link. The visitor is 
totally unconcerned about the where or how or the me- 
chanics — the information appears like magic. To the end 
user, the systems appear unified and integrated even al- 
though they are in reality separate. 

This approach lends itself well to enterprise integra- 
tion, but until recently the difficulty has been as always 
the API's and accessing the internal content of documents 
reliably. The Systems architect has been dependent on 
the vendor providing hooks into the legacy system, and 
often this is very expensive, specialized or limited. How- 
ever, with Apache Solr, these boundaries can be crossed 
and intelligent search made available across the enter- 
prise (Figure 4). 


So what are Solr, Tika and Nutch? 
Apache Solr is an enterprise grade search platform which 
emerged from the Lucene project. Highly scalable its ma- 


Apache Solr examples and JAR/WAR files 


http://lucene.apache.org/solr 


jor features include powerful full-text search, hit highlight- 
ing, faceted search, dynamic clustering, database integra- 
tion, and the ability to index a wide range of documents 
and meta-data from disparate file formats. 

The Apache Tika toolkit detects and extracts meta-data 
and structured text content from various documents using 
existing parser libraries. 

Apache Nutch is a web crawler used to pull content from 
websites. Robust and scalable, Nutch can prioritize what 
pages are fetched first. 

The biggest challenge to implementing Solr effectively 
is designing a suitable schema that is powerful enough 
to answer queries yet flexible enough to be extensible. 
Solr is not an RDBMS, it excels at language manipula- 
tion, ranking and faceting as well as parsing and extract- 
ing content and meta-data from a wide variety of sources 
when used with Apache Tika and Nutch. This requires a 
different approach when it comes to system design from 
that of database and relational architectures. 

In this series of articles, we will build a Solr 4 search en- 
gine under FreeBSD 9 (clean install) with the latest version 
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of Tomcat 7. We will look at 


Let's Get Started 

First download diablo-caffe- 
freebsd/-i386-1.6.0 O/7- 
b02.tar.oz2 and accept the 
licence for Diablo Version 
1.6.0-0. Place this file in / 
usr/ports/distfiles, 

Then download the fol- 
lowing files for the Solr in- 
stall (using the most con- 
venient mirror) and place in 
temporary directory some- 
where (€.g. /tmp/soir) (Ta- 
ble 1). 

As root, bring the ports 
tree up to date and install 
Tomcat 7 from source: 
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portsnap fetch update 

cd /usr/ports/www/tomcat7 

make install BATCH=YES 

In -s /usr/local/apache-tomcat-7.0/logs /var/log/tomcat7 


This will download the source code, building Tomcat 7 
from scratch and will take some time so go and grab a 
coffee. When the sarcu=ves switch is included to make, it 
should run unattended with the default settings provided 
you have Diablo Version 1.6.0-0 in disttfiles. 

Once complete, add the following line to /etc/rc. conf US- 
ing your favourite editor to ensure Tomcat starts correctly 
on reboot: 


cvomeaty endble="YES” 


lf you want to use the Tomcat web manager via a web 
browser, add the following lines to /usr/local/apache- 
tomcat-7.0/conf/tomcat-users.xml before the </tomcat-users> 
tag (Use a strong password in an production environ- 
ment): 


<role rolename="manager-gui"/> 
<user username="tomcat" password="tomcat" 


roles="manager-gui"/> 


Start tomcat: 


Listing 1. /nstalling Tomcat & Solr 


puew) locally ete) re-d/ tomcat] ss top 

cd 7 emp/ soln 

baie XVZe (apacne-solr—4 70) 0-BERA tgz 

cd /tmp/solr/apache-solr-4.0.0-BETA/example 
Ce =hesolm home 

cp -r exampledocs /home/solr 

mv /home/solr/bin /home/solr/collectionl/bin 


medic /home/solr/ collection, lib 


chown -R www:www /home/solr 


Cad. 23) Gist 


<p @aeie— Shia 


find /tmp/solr/apache—-solr—4.0.0-BETA -iname “*.jar” -exee cp —v {} 


UnmZINe “apache—-solr—4 0 O—BEtAlwar —-d /usr/ local/apache—tomeat—7 .0/webapes/solx 
chown -R www:www /usr/local/apache-tomcat-7.0/webapps/solr 
touch /usr/ local apache -tomeat— 1.0/7 cont) Carallina, localhost, Soll sxmil 


chown www:www /usr/local/apache-tomcat-7.0/conf/Catalina/localhost/solr.xml 


Listing 2. Adding correct Java library path to Tomcat solrconfig.xmI file 


pe 
“lab ydir =". ./dist/” regex—“apache -solr—celll—\d.*\.jar”’ 7> 
@lilpidite="s./ Contrib, extraction; bib” reqex=" 2" \ jan” =/> 
“lib div—"Fe/dist/~ ~reqex—"apache-solr-clistering—\de*\- jar” > 
“ib vdic=".2/ contrib, clustering, lib; iregqex="24 \\.qar” /> 
ii bidiv="2./dilet/” regex— apache solr —langrtd>\d.* ajar ¥/ > 
<li ir) yy contriby Langid)/ IMib/aregqex—" 4 \ eer” > 
Siiprdir= (7 dist)? “egex—apache-csolle-velocity—=\d. \ejan > 
<li dir=". s/contrib; vellocity/ lib” regex—".4*\.yar” 7> 
meee 
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Listing 3. Result returned for “Enterprise” 


This XML file does not appear to have any style information 


associated with it. The document tree is shown below. 


V-vesponse- 

V¥<lst name="responseHeader”> 
<int name="status”>0</int> 
<int name="QTime”>4</int> 
Vv<lst name="params”> 
<str name="q">Enterprise</str> 
“sir Mane— we ox </ out 
Gy) Meee 

</a> 

V<result name="response” numFound="1”" start="0"> 

V-<doc> 
<str name="”id”>SOLR1000</str> 


<str name="manu”>Apache Software Foundation</str> 
V<arr name="cat”> 
<str>software</str> 
<str>search</ stir 
<7 aera 
V<arr name="features”> 
ES Gree: 
Advanced Full-Text Search Capabilities 
using Lucene 
<<) Seie 
<str>0ptimized for High Volume Heb Traffic</str> 
<str>Standards Based Open Interfaces - XML and 
FMM IP e/tctg ae 
<str>Comprehensive HTML Administration 
Interfaces</str> 
Vere 
Scalability — Hihcrenk, Replication to other 
Solr Search Servers 
</Str> 
Veer 
Flexible and Adaptable with XML configuration 
and Schema 
oie: 
Vere. 
Good Unicode support: héllo (hello with an 
accent over the e) 
<<“ Seu 
</ an 
<float name="price”>0.0</float> 
KSicie meiie= (aes 0 US DS) sice> 


<int name="popularity”>10</int> 


<bool name="inStock”>true</bool> 
“Gabe Meme— IMicUvcr ondarercde 2006-0 1 
Lj T007007004</daee- clong name= "9 
version “>1411798689903542272</long> 
<G/ lore> 
</result> 


</response> 


Listing 4. Search for “Video” 


This XML file does not appear to have any style information 


associated with it. The document tree is shown below. 


V-vesponse. 
¥<lst name="responseHeader”> 
<int name="status”>0</int> 
<int mame—"OTime”’>23</ int> 
v<lst name="params”> 
<str name="fl”>name, id, score</str> 
<str name="q">video</str> 
<<) lice: 
<p lice 
V<result name="response” numFound="3” start="0" 
maxScore="0.500039"> 
V<doc> 
<str name="id" >MAI47 il/A</stxr> 
<str name="name”>Apple 60 GB iPod with Video 
Playback Black</str> <float 
name="score”>0.500039</float> 


<<, oe> 


<str name="id”>EN7800GTX/2DHTV/256M</str> 
<str name="name”>ASUS Extreme N7800GTX/2DHTV (256 
ME) </sica- 
<float name="score”>0.3849302</float> 
<j Clolee: 
¥<doc> 
“Ste Neme=— 1d —L00-435005—/ str> 
<str name="name”>ATI Radeon X1900 XTX 512 MB PCIE 
Video Card</str> 
<float name="score”>0.3849302</float> 
</doc> 
</result> 


</response> 
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/usr/local/etc/rc.d/tomcat? onestart 


You should see the following screens at. nhttp:// 
yourserverip:8080 and 
where “yourserverip” is the external IP address of the 
your FreeBSD install (Figure 5 and Figure 6). 


http://yourserverip:8080/manager 


Installing Solr 
Initially, we will run Solr in single core mode without clus- 
tering. For this demo we will use the example documents 
and schemas from Collection1 supplied by Apache (List- 
ing 1). 

Edit /usr/local/apache-tomcat-7.0/conf/Catalina/localhost/ 
solr.xml aS follows: 


<?xml version="1.0" encoding="utf-8"?> 

<Context docBase="./solr" debug="0" crossContext="true"> 

<Environment name="solr/home" type="java.lang.String" 
value="/home/solr" override="true"/> 


</Context> 


Edit /home/solr/collectionl1/conf/solrconfig.xml to reflect the 
following: Listing 2. 

Finally, reboot the server to test that everything will 
come up at boot: 


reboot 


Solr should now be running at http:/vourserverip:8080/ 
solr (Figure 7 and Figure 8). 


Indexing and Retrieving Data 


su 
cd /home/solr/exampledocs 


pkg_add =r curl 
Then edit the post.sh file to show: 
URL=http://localhost:8080/solr/update 
Index two documents: 
sf POSS. Sole. em: mont or xml 


You should see the files being posted. 

Now visit htto://vourserverip:8080/solr/collection 1/select?q 
=Enterprise&wt=xnil. 

You should see your document returned in XML format 
(Listing 3). 

Index all the XML docs in the examples directory: 
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References and further reading 
Apache Tomcat - http://tomcat.apache.org/download-70.cgi 
Apache Solr - http://lucene.apache.org/solr 
Apache Tika — http://tika.apache.org 
Apache Nutch — http:/nutch.apache.org 


.fpost.sh *.xml 


Now search for video only displaying the name, id and 
score field: 


http://192.168.0.127:8080/solr/collectionl/select?q=videoéfl 


=name,id, score 


You should see 3 results returned in XML format (List- 
ing 4). 

Finally, visit http://192. 168.0.127:8080/solr/browse. 

You should see faceting in action. If you encounter fa- 
tal tomcat errors (SEVERE SolrDespatchFilter etc), check 
that the *.jar files from contrib and dist trees have been 
copied across and that <1ib air /> setting is correct. 


In the Next Article ... 
We will look at synonyms, stemming and the data handler. 
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oper, systems integrator and IT manager. He has moved on from 
CP/M and nixie tubes but keeps a soldering iron handy just in 
case. 
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PostgreSQL 


Partitioning (Part 2) 


In the last article data partitioning was introduced and an 
application example consisting of a forum database was 
used to explain how to partition tables, migrate data and 
route queries to the right data set depending on the forum 


Te Ml 


post's “category” and “timing’. 


What you will learn... What you should know... 
« Howto use tablespaces to handle database data ¢ basic shell commands 
« how to implement partitioning that exploits tablespaces ¢ basic PostgreSQL concepts 


¢ partitioning concepts explained in the previous article 


n this paper readers will further extend the application 
scenario presented in the previous paper, implement- Listing 1. The initial situation for the examples 
ing a physical partitioning that keeps tables and data 
in separate storage devices. All the examples shown here bsdmag=> SELECT relname, reltuples FROM pg class 
have been tested on a PostgreSQL 9.1 cluster running on WHERE relname like ‘thread%’ AND relkind = ‘r’ ORDER BY 
a FreeBSD 8.2-RELEASE machine; see the previous ar- relname; 
ticle in this series for details about the application scenario relname | reltuples 
and how toreproduce it, nnn nnn nena nn---------- $o---------- 
thread | 0 
Improving the Partitioning thread_hw | 0 
In the previous article readers saw a simple database, thread hw year1991_—| 29014 
called forumdb, populated with around 4 million tuples a 
representing forum posts, contained in a main thread ta- thread hw year2004_ | 553 
ble. This table was then partitioned first into a per-catego- thread _kern | 0 
ry table (e.g., thread_net) in order to group posts by their thread kern year1993 | 43621 
category; subsequently the data was partitioned further = 
based upon the year a post created. Of course, ad-hoc thread kern year2012 | 13255 
constraint checking as well as triggers and rules were built thread misc | 0 
to route insert queries and to avoid data corruption (i.e., thread misc year1990 | 58282 
storing a post into the wrong table). ae 
The situation could be summarized as shown in List- thread misc year2012 | Lg RO 
ing 1 UMeeacd Wet | 0 
While this partitioning is effective, it probably does thread net _yearl992 | 72943 
not achieve the overall goal of allowing for the high- aoe 
est possible performance of an interactive forum. It is thread net year2012 | 22165 
worth noting that insertion of new posts will always be 
performed on the last per-year table of each category; 
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excluding application bugs and forum recovery, posts 
that are in the past will not be changed and no post in 
the past can be added. On the other hand, old posts 
could be the needed for performing queries, and there- 
fore it is not possible to just discard such posts. This 
scenario is therefore asymmetric: the last per-year table 
of each category will be actively used for both queries, 
updates and insertions, while the other per-year tables 
will be used exclusively for queries. In such a scenario 
it is therefore possible to give the “current year” table 
of each category priority over the other tables, so that 
queries affecting the current year are optimized in some 
way to complete faster than queries to other tables. A 
possible way to achieve this is to use different storage 
devices for different tables, so that the current-year ta- 
bles are stored on faster disks while other tables will be 
stored on slower disks. The feature that allows Post- 
greSQL to have different storage systems is referred to 
as tablespaces. 


Introduction to Tablespaces 

Tablespaces are storage locations, file system hierar- 
chies, that can be used to store database objects (main- 
ly tables and indexes). As explained in the first article of 
this series, PostgreSQL stores all objects within a file 
system hierarchy identified by the environment variable 
spcpata, IN particular the secpara/base Contains all the da- 
tabases and their data in files named after the OID of the 
table/object itself (with a few exceptions). 

Tablespaces represent a way to “escape” the spcpara 
directory allowing the cluster to use extra disk stor- 
age, different speed and architecture disk storage and 
even different file systems. Several scenarios are pos- 


Box 1. Using a memory disk for experiments 
During the writing of this article the author used a memo- 
ry disk (vnode backed) to simulate a very fast disk attached 
to the machine and mounted at /postgresql/fast-disk. The 
following are the steps required to reproduce the simulation 
with a memory disk identified as /dev/md10: 


# touch /postgresql/memory disk.md 

# dd if=/dev/zero of=/postgresql/memory disk.md bs=1M 
count=50 

# mdconfig -a -t vnode -f /postgresql/memory disk.md -s 
SUI) 0 

# mdconfig -l -v 

md10 vnode 

# newfs /dev/md10 

# mkdir /postgresql/fast-disk 

# mount /dev/md10 /postgresql/fast-disk/ 


50M /postgresql/memory disk.md 


The usage of a memory disk is beyond of the scope of this ar- 
ticle, please refer to the operating system documentation. 
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sible: not only most frequently accessed tables can 
be stored on a faster mass storage device, but al- 
so indexes can be placed elsewhere. Besides storage 
speed, tablespaces allows also for a storage capacity 
scale up. 


Defining a Tablespace 

Suppose that the database machine has a fast storage 
disk mounted at /postgresql/fast-disk. IN order to make 
PostgreSQL selectively use such disk a new tablespace 
has to be defined. Defining a tablespace requires the cre- 
ation of a directory with the right user and permissions so 
that only PostgreSQL can use it: 


# mkdir /postgresql/fast-disk/forum tablespace 
# chown pgsql:pgsql /postgresql/fast-disk/forum tablespace 
# chmod 700 /postgresql/fast-disk/forum tablespace 


No further disk initialization is required for a tablespace 
to be used. Having defined the storage location to use 
as tablespace it is possible to inform PostgreSQL about 
such location using the create tastespace Command (as 
database superuser): 


forumdb=# CREATE TABLESPACE ts forum 
OWNER forum 
LOCATION '/postgresql/fast-disk/forum tablespace'; 


The tablespace is called ts forum and “points” to the / 
postgresql/fast-disk/forum tablespace directory; MOreover 
the ownership of this tablespace is granted to the Post- 
greSQL user forum. What happens on disk is that a sym- 
bolic link is created from the cluster storage directory 
to the tablespace location, in particular the secpata/pg _ 
tblspc directory contains links to all the tablespaces de- 
fined in the cluster: 


~> ls -l /postgresql/clusterl/pg tblspec 
1 pgsql pgsql 38 Apr 19 13:49 76871 -> / 


postgresql/fast-disk/forum tablespace 


Thanks to the linking mechanism PostgreSQL can reach 
all the tablespace storage locations without having to 
“escaping” from the secpara directory. 

Having defined the tablespace is now possible to use 
it to store database objects, in particular tables and their 
data. The create tasne Command has the special option 
TABLESPACE that allows the specification of a tablespace to 
use; therefore issuing a command like: 


CREATE. TABLE thread net 2012 ( . ) TABLESPACE ts forum; 
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Listing 2. A stored procedure to change tablespace of the 
current-year tables 


CREAVE OR REPEACH PUNCIION Migrave tables roy 
tablespace () 

RETURNS integer 
AS 
SBODYS$ 
DECLARE 

CUGEEME TeCakegory, CaLcdOLry obowE yee, 
CUv Bent yeas Piece a 
Megrazed tables nS Ce yy 


BEGIN 
mibbenceicecl celelies -— (0: 


SELECT EXTRACT year PROM current date ) 


INTO Clie Kenr syeaic; 


-- iterate over each category 

BOR NCUMMeM Enea weet a1 N aol abe ies 
HROMPcabeqon, 
ORDER BY 1d 
LOOP 


EXECUTE ‘ALTER TABLE * 
Be viclavetcycto Sell ereuses renee 
Cacedory id WN" tyean {|mecurvents 
year 


jy] oat  PAbIN SPACE Fe FOLUM 


Muigtavecd eaoles  ; — tugiatee alles =r 
1; 
END LOOP; -- end of the category 


ab Cleng sNe ake he 
INVSMIUISIN, bkepaaieiecl Welle lens - 
END; 


SBODYS 
LANGUAGE plpgsql; 


will result in the creation of the thread net 2012 ta- 
ble that will be stored in /postrgresql/fast-disk/forum _ 
tablespace Storage hierarchy (ts_ forum tablespace). Be- 
cause PostgreSQL allows the storage location of a table 
to be defined when the table is created, partitioning can 
be split across different hierarchies. In the above exam- 
ple however the tables are already in place and cannot 
be re-created. Fortunately, PostgreSQL allows the mi- 
gration of a table to another tablespace using the atrer 
TABLE SET TABLESPACE COmmana4d. It is therefore possible to 
build a simple stored procedure that iterates over each 
category and migrates the current year table (see List- 
ing 2). The result will be that each 2072 table (the current 
year) will have a tablespace while all the others will not: 


forumdb=> \d thread net year2012; 


Inherits: thread. ner 


Tablespace. “te Forum” 


This means that, on disk, under the tablespace hierarchy 
there will be a set of files, each one named by its OID 
(see the first article in this series). Given that the 2012 
tables have the following files on disk: 


Box 2. How to know which tablespaces are 


available 

It is possible to see which tablespaces are available within 
a cluster using the special command \qdb, that reports both 
the location, the tablespace name and the owner of the ta- 
blespace itself: 


forumdb=> \db 
List of tablespaces 
Name | Owner | ILGIGELE LOI 
os ene oe een Be ea ae a rn ee Oe et RE ee eae ee 
oc elsmetlbe || jocstell | 
pg_global | pgsql | 
tse eorum (Peorume Ny Postarecgqi tast—disi, ronums 


tablespace 


Another way to collect tablespace information is the usage of 
the pg_tablespace catalog: 


forundb—> sPlbCl spename, spelocation FROM pgs 


tablespace; 
spcname soclocation 
pai re arenietyet weeyecpets Eye at cas ea ak ta Ee aS 
po deta: 
pg global 
ies. forum /postgresql/fast-disk/forum tablespace 
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Listing 3a. The create_category_tables stored procedure that will 


: , . RAISE LOG ‘Generating time tables from 
exploit the “fast” storage tablespace 


ie) 


Vea 6 LO year so 7, Cucmenl year, 


CREATE OR RERUACE FUNCTION ereave Careqory cables) 


RETURNS integer WHEE Seulpreir year o<— CubreiE Max year 
AS LOOP 
2BODYS RAISE LOG ‘Creating sub-table for 
DECLARE Vea > 7 scUMeroimy EyCalc; 
Cur cemepearegony categorysrowtype; CUM @emE yy can eOn cineca an inner 
Greated tables integer; Vea EROMMeiureny eavegouy. sinecy)s 1 
CUGheME Ecole mame text, CUMeGemr yea = il, 
CUlieme tm wy cee integer; 
Cur eenE query Pex, 
UNgIcee Wisc Ee clccl< | JLMeEeioue Saher Cheat Tih Le NOl ios 
CUrKenE Max year IMME eGet Ehuead ) 
BEGIN WS eteiigia cuaties cereiciefouay/ navel) |||. 
Year’ «||| Current year 
created tables := 0; Tae Sk: ae 
-- iterate over each category (ee CHECKIN = 
BOR eUrBeme neairegomy oN SS tikE Gil” || * EXTRACT (year FROM 
FROM category Publ ishedson) = 
ORDER BY id CUNRe MEN Vea 
HOOE ee 


‘ PRIMARY KEY (ok), * 
~ FOREIGN KEY (Category pk) 


-- build a dynamic query for creating the REFERENCES category(pk), ‘ 
table [>> FOREIGN KEV (antinox jek) 
~-EARCUIET DROP TABLE -tnecad ’ "|)| REFERENCES author(pk), ‘ 
CuGrenb cCakegory. 1d; >) UNTOUR( Ere, mac)“ 
EXECUTE ‘CREATE TABLE IF NOT EXISTS ye INTER Ss 
iieach ||P eukneniencakogonmy ard ey ehimeaq a | eureemty 
Up (elbCr(Geneco cyclic =» || category.id 
CUMbeMEncakegomy pias yn 
[> PRIMARY KEY (pk), ~ A ies ee 
(>> FOREMEN VKEY (Category = pk) ENDO -culerent query, 
REFERENCES category(pk), ‘ 
PS SS ORE TENT Wi (auiior pl) -- is this year the current one? 
REFERENCES author(pk), ‘ Do we have to use 
Pe SUN OUE: (eae = imac: )s) -- the fast tablespace? 
i) SINE RIES:  Coleead).. 7 > IE CuUrrenr year — Currene max) 
year THEN 
ereaved tables ;— crcated tabiles: + i, CUEPERE @qulery {— Current 
query |||) TABLESPACE ts torum | 
-- compute the current year BND IF; 


CUrreME year). — EXTRACT (year FROM 


CUGBEME eC aAbedony. simce )7; CUbteiiE query. = Current iqueny 
DET PAETRACT ( year PROM current adare)) (ids eae 
INTO Cul Belin Max a year, EAE CUTIE “current yqueny,; 
CUBBemb Sycem = Cumeene yeama. Aly 
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Listing 3b. The create_category_tables stored procedure that will 
exploit the “fast” storage tablespace 


END LOOP; -- end of the per-year-while 


END LOOP; -- end of the category iteration 


RETURN created tables, 


BID: 
SBODYS 
LANGUAGE plpgsql; 


Listing 4. A stored procedure that migrates indexes by their names 


CREATE CR SP EACH SUNCTICh Mia Gmate indexes. ro. 
tablespace () 


RETURNS integer 


AS 

SBODY$ 
CUmGemE  Cavegory categorysrowtype; 
ClmBentayean integer; 
migrated indexes integer; 

BEGIN 


mliLopesnusicl tales FS 10): 


DELECT EXTRACT ( year PROM current dare } 
INTO CUIGeneE Syeac; 
-- iterate over each category 


MOIR, WeWhigig Siete este clorena VIN SinIaCay se 


FROM cabeqory 
ORDER, BY id 
LOOP 


‘ALTER INDEX * 
WS Selsig cree (i) 


EARCUTE 
CUI EemEs 
category.id 

|| > year’ UP ethertsintc Seu 
DP) apkey’ 
(ie sh TABIRS PACH Gs Utonum 1; 


SALTER INDEX * 
[ie theeacde! =i) 


EXECUTE 
Ula e mes 
category.id 

li *Syeax? U ‘ethereine cen 
|| * tid mid key’ 
|| * SET TABLESPACE ts forum %; 


tMgbhaeoC WaCexcs, | ~wimeheabed mide xesi 
ay 
END LOOP; -- end of the category iteration 
RETURN migrated indexes; 
END; 


SBODNs 
LANGUAGE plpgsql; 


Box 3. How to quickly set up (again) the 


database 

To build up the database as in previous article, and in order to re- 
peat the examples shown here, it is possible to issue the follow- 
ing commands (being connected to the forumdb): 

DROP TABLE IF EXISTS thread net CASCADE; 


DROP VABEE iE exists) theead muse CASCADE, 
DINOE Nello Ie ly Ledhs iwiigcetiel Vee (Ceva e/a )B lah 
DROP TABLE UF Exists theead mise CASCADE, 
DROP TABLE IF EXISTS thread CASCADE; 

DROP FABIE UF Ex SiS vaubnior, 

DROPS TAP Tie hx > Lo seake goin, 

DROP VIEW TF EXISTS vw thread; 


\i 01-forum-database-initial-setup.sql 
Mi OZ-frumet Lom populate. cell 
DELECT populace, forum (); 


\i 03-function-create-category-tables.sql 
SMMHGCIE Tees tS Tectrocioie inclollios (|): 

V0 4 iirc tronese 

SEMAN iikgieence elas els) 2 

\i 05-thread-table-rules.sql 

SHC Teieeclins eencecioey seulles ()) 

MEO —perrreroming— tame. sell 

SEMI CI whe gels wiesecls: loyy Cawioe@iey cual winnie) 6 
SELECT MicSeis CemegOey Lule wielecieics |) 3 
VACUUM FULL ANALYZE; 


Please note that all the scripts come from the GitHub reposi- 
tory (see the references) and are contained in the bsdmag/05- 
partitioning directory. The whole process could require more 
than 20 minutes to complete, depending on the speed of the har- 
dare. 
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On The Web 

- PostgreSQL official Web Site: http://www.postgresql.org 
ITPUG official Web Site: http:/www.itpug.org 
PostrgeSQL Table Inheritance Documentation: http://www. 
postgresql.org/docs/current/static/ddl-inherit.html 
PostgreSQL Tablespace Documentation: http://www.post- 
gresql.org/docs/current/static/manage-ag-tablespaces.html 
GitHub Repository containing the source code of the ex- 
amples: https://github.com/flucal1978/fluca-pg-utils 


forumdb=> SELECT relname, relfilenode 
FROM pg class WHERE relkind='r' AND relname like 
"ihibead: s_ yeerz0lZ2 "5 


relname | reliilenode 
a ee eee eee ee [een eee ee ee 
thread hw year2012 89461 
thread _kern year2012 89464 
thread Msc. yearZ012 89467 
thread net year2012 89470 


the tablespace hierarchy will contain files with the same 
names. 

In the case where data partitioning has not yet been 
completed, and the per-year tables have therefore not 
yet been created, it is possible to change the stored pro- 
cedure which creates the tables (see Listing 3) so that 
when the table being created is that of the current year, 
the “fast” storage will be used. 

Atablespace can be used also for storing indexes, there- 
by improving speed of indexed access to the data. Since 
the example tables all have only two indexes (the primary 
key index and a unique index on the couple ¢ia, mia) it Is 
possible to build a stored procedure that will also migrate 
the indexes to the new tablespace using the ALTER IN- 
DEX SET TABLESPACE statement (see Listing 4). 
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The BSD Certification Group Inc. 
(BSDCG) is a non-profit organization 
committed to creating and 
maintaining a global certification 
standard for system administration 
on BSD based operating systems. 


BSDA: Entry-level certification suited for candidates 
with a general Unix background and at least six months of 
experience with BSD systems. 


BSDP: Advanced certification for senior system administrators 
with at least three years of experience on BSD systems. 
Successful BSDOP candidates are able to demonstrate 

strong to expert skills in BSD Unix system administration. 


@ WHERE CAN 1GET CERTIFIED? 


We're pleased to announce that after 7 months of 
negotiations and the work required to make the exam 
available in a computer based format, that the BSDA 
exam is now available at several hundred testing centers 
around the world. Paper based BSDA exams cost $75 USD. 
Computer based BSDA exams cost $150 USD. The price of 
the BSDP exams are yet to be determined. 


Payments are made through our registration website: 
https://register.bsdcertification.org//register/payment 


ap WHERE CAN I GET MORE INFORMATION? 


More information and links to our mailing lists, LinkedIn 
groups, and Facebook group are available at our website: 
http://www.bsdcertification.org 


Registration for upcoming exam events is available at our 
registration website: 
https://register.bsdcertification.org//register/get-a-bsdcq-id 
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Hardening FreeBSD 


with TrustedBSD and Mandatory Access Controls (MAC) Part 3 


Most system administrators understand the need to lock 
down permissions for files and applications. In addition to 
these configuration options on FreeBSD, there are features 
provided by TrustedBSD that add additional layers of 
specific security controls to fine tune the operating system 


for multilevel security. 


What you will learn... 
¢ Configuration of the mac_bsdextended module. 
¢ Howto use the ugidfw utility 


tensions have been included with the default in- 

stall of the operating system. By default, this func- 
tionality is disabled and requires support to be compiled 
in or kernel modules to be loaded at boot time. For the 
purpose of this article, support will be loaded in with ker- 
nel modules already available with FreeBSD 9. Part 3 of 
the TrustedBSD series will cover the basic configuration 
of the mac _bsdextended module. 


S ince version 5.0 of FreeBSD, the TrustedBSD ex- 


Warning 

Incorrect MAC settings can cause even the root user to 
not be able to login to the system. Be sure to run these 
tests on a VM or test machine to avoid any issues with 
production systems. This article assumes that a fresh in- 
stall of FreeBSD 9.0 with a separate file system called 
“data” has been performed before continuing. 


As in the previous articles, a certain set of users will 
help to illustrate how to use mandatory access controls 
(MAC) to fine tune access to specific file system objects. 
Listing 1 shows the layout of the users and groups setup 
on a separate file system called “data” and how to create 
them. There is a project to enable discretionary files but 
for this article the focus will be on file system restrictions. 

The mac _bsdextended Module creates essentially a file 
system firewall that has a syntax similar to the ipfw fire- 
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What you should know... 
¢ Basic FreeBSD knowledge to navigate the command line 
¢ Familiarity with loader.conf to enable kernel modules at boot 


wall. In order to load the module on boot, add the following 
tO /boot/loader.conf aS detailed in Listing 2. 

Once the system is rebooted, the ugidatw utility will be 
able to make changes using the loaded module. Listing 3 
shows the default output from using the ugiatw utility which 
should not list any rules. The sysctl value should show 
that mac _bsdextended is enabled. 

Unlike the previous modules, mac bsdextended does not 
require changes to policy labels to enforce the access 
controls. Everything is configured using the ugiatw utility 
with the rules being evaluated in order. This utility high- 
lights the ability to restrict access to objects to authorized 
subjects, which is an important part of mandatory access 
controls. For this example, user2-reg directory will be 
changed so that only user2 has access to the directory for 
which user1 would normally have access through group 
permissions. Listing 4 shows the usage Of ugiatw, with the 
output from user1 trying to access the directory before 
and after the change. 

With the group permissions allowing user1 to access 
the directory, setting a rule to only allow a user with a uid 
matching the directory ownership overrides the standard 
group permissions. Listing 5 shows an additional rule to 
examine the gid of the subject. However, because of the 
previous uid rule, the new gid rule is not evaluated. 

In order to open up the permissions to any member of 
the user-reg group, the rule order must be changed. List- 
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Listing 1. Directory setup on FreeBSD for several users called /data 


# mkdir -p /data/userl-reg 

# mkdir -p /data/user2-reg 

# touch /data/userl-reg/secret-order.txt 
# touch /data/user2-reg/secret-order.txt 
7 OW WiSer ead =n Wiser! ss / bin cen. om 

# pw tser ad@ —a user? —s /bin/csm =m 

# pw group add user-reg -M userl,user2 

# passwd userl 

Changing local password for userl 

New Password: 

Retype New Password: 

# passwd user2 

Changing local password for user2 

New Password: 

Retype New Password: 

# chmod -R 770 /data/userl-reg/ /data/user2-reg 
# chown -R userl:user-reg /data/userl-reg 
# chown -R user2:user-reg /data/user2-reg 
# ls -ltra 
oral 20 
Ciwrwxr—x 2 POOE operator SIZ seo 2 1240 snae 


(QUENT ate ace) mee Alber a6 (6.16 wheel 1024 Sep 2 12:40 


drwxrwx--- 2 userl user-reg D2 Sep 2 2558 userl— 
reg 

drwxrwx--- 2 user2 user-reg DZ Sep. 2 i258 user7 — 
reg 


Gian gie oa SGI wheel OE Seen IL ache 
# groups userl 
userl user-reg 
# groups user2 


user2 user-reg 


Listing 2. Loading the mac_biba module on system startup 


7 seeNO ae sOsdexrended Noad— “Her | 9/loon/ loader, comm 
# reboot 


Listing 3. Output from ugidfw with validation the module is loaded 


# sysctl -a security.mac.bsdextended. enabled 


security.mac.bsdextended.enabled: 1 


# ugidfw 
wsage: ugidiw add ‘subject [nor] [urd uid) [oid did] 
object (incr) [uid uid) \ 
[gid gid]] mode arswxn 
ugidiw list 
ugidfw set rulenum [subject [not] [uid uid] [gid 


gaat | 


[gid gid]] mode arswxn 


[object [not] \ 
[otirel auligel 
ugidfw remove rulenum 

# ugidfw list 


0 slots, 0 rules 


Listing 4. Using ugidfw to restrict access to the the user2-reg 
directory 


# echo “TooManySecrets!” > /data/user2-reg/secret-order. 
EXE 

# su - userl 

S$cd /data/user2-reg/ 

scat secret-order.txt 

TooManySecrets! 

SeEX1t 

logout 

# ugidfw set 1 subject uid userl:user2 object uid 
psec sisee? wihesys jeleice | Uuiel Oi 
subject mode n 

# su - userl 

S$cd /data 

Ase = eel 


ls: user2-reg: Permission denied 


Uotal 

ie wixcewace— x ZOO le operator SIZ sep 2 12:40 esnap 

GieWeere = ir Zr OOie wheel 1024 Sep 2 12:40 

CAVE = cin Ss) eC lene wheel Sl AS Siem 2 Plas ore 

Gry xrwx——-— 2 userl user-reg SEA eceo. (253 PZ oruser iS 
reg 
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Listing 5. Using gid to allow access to the object. Rule 1 triggers for 
user! when trying to view the user2-reg directory and vice versa with 
user2 trying to view user1-reg 


# ugidfw set 2 subject uid userl:user2 object uid 
Use suse? mlesys /Oere | Cael Oi 
subject mode n 

# ugidfw list 

S silhoes 2 sulle 

1 subject uid userl:user2 object uid userl:user2 filesys 
/data ! uid_of subject mode n 


2 subject uid userl:user2 object uid userl:user2 filesys 


/data ! gid_of subject mode n 
# su - userl 
cd /data 
Olsee lieied 
ls: user2-reg: Permission denied 
pele kG 
drwxrwxr-xX A SOONG operator 512 Sep: (2° 12:40 ssnap 
1024 Sep 2 12:40 .. 
DZ Sep 2 78. 


512 Sep 2 13:25 userl-reg 


drwxr-xr-x 21 root wheel 


dYrwxr-=xr-x Ss) cole wheel 


drwxrwx--- 2 userl user-reg 
Sexit 

Logout 

# su - user2 

2e0 7) Gata 


Sls -ltra 


ls: userl-reg: Permission denied 


pO eer eaikG 

GEWwxXGWwx =x ZOO operator ol? seo 2° 12740 snap 

Gewese ee OOF wheel i244 sep a2 W240. 

Geir Kr ax OF ZEGIOIE wheel SE Seow 2 elas oe 

CuwxrWwx=—— 2 user2 user-reg Dl Sep. 2 si 26 wserZ— 
reg 


Listing 6. Rule for allowing user! and user2 to access anything with 
the gid of user-reg 


# ugidfw set 1 subject uid userl:user2 object uid 
uSseel eusen, mlesyes j/oatea | oie Oi 
subject mode n 

Se siccs, | rules 

1 subject uid userl:user2 object uid userl:user2 filesys 
/data ! gid_of subject mode n 

# su - userl 

cd /data/user2-reg/ 

scat secret-order.txt 


TooManySecrets! 


) 
© 


ing 6 shows the rule to allow user1 and user2 to access a 
directory if their group id matches. 

The examples in this article used directories as an easy 
way to highlight the usage of the ugiatw utility and the mac_ 
bsdextended Module. Moving beyond this example, the per- 
missions could be extended to go beyond the data file 
system and to all file systems to restrict user1 and user2 
across the operating system. These controls allow for an 
additional layer of security in the case that a user creates 
a file or directory that does not restrict access. With the 
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uid being set to that of the user with the file system fire- 
wall, access restrictions can be uniform so that the indi- 
vidual user must give access to the file. In later articles, 
the MAC modules will be combined to present different 
layers of security and to help with classifying information. 
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Interview with 


Jeroen van 


Nieuwenhuizen 


Jeroen van Nieuwenhuizen was the chair of the EuroBSDcon 
2011 organizing committee. Currently, he is one of the 
members of the EuroBSDcon Foundation board. He came in 
contact with Unix in 1997 and started to work with the BSDs 
in 2002. In his daily life Jeroen works as a Unix Consultant for 


Snow B.V. 


Can you tell us what the EuroBSDcon 
Foundation is about? 

Jeroen van Nieuwenhuizen: The EuroBSDcon Founda- 
tion is an idea that existed for several years already. After 
the 2011 conference, Fred Donck, Paul Schenkeveld and 
| decided we should go the extra mile and realize it. 


What is the mission of the EuroBSDcon 
Foundation? 

JvN: The goal of the EuroBSDcon Foundation is to make 
it easier to hand over experience between years. 

Also, financial resources can be transferred from one 
conference to another. If one conference has money left 
at the end, it can be transferred to next year’s conference 
to cover some potential future loss. Additionally, the Foun- 
dation can also help with infrastructure when necessary. 
For example, the EuroBSDcon Foundation is handling the 
registration for the EuroBSDcon 2012. 


Where did the idea of a EuroBSDcon 
Foundation come from? What made 2011 the 
year you went the extra mile? What happened? 
JvN: | don’t know exactly who first came up with the idea 
of the Foundation, because the idea was around before | 
got involved. In Karlsruhe in 2010 the idea was put back 
on the agenda and the idea was to have the Foundation 
ready to support the organization of EuroBSDcon 2011. 
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However, due to the amount of work to setup a foundation 
and all the legal issues involved it proved too difficult to 
get it up and running in time. During and after the EuroB- 
SDcon 2011 we realized we were facing the same prob- 
lems as earlier years and decided to move forward. 


How do they pass knowledge from one event 
to the other? Wiki? White papers or something 
else? 
JvN: One of the issues we ran into this year was that the 
Foundation started after the 2012 organizers and a lot of 
the 2011 experience isn’t documented in a format that 
would be directly useful for other organizers. So currently 
most knowledge is passed by email, IRC and phone. 
| like to look at 2012 as an ‘experimental’ year of how lo- 
cal organization and the Foundation should work together. 
Some improvements that we can make are better templates 
for budgeting, sponsor benefits and the overall planning. 
Furthermore, we are working to get infrastructure, like a 
wiki and version management, in place. 


Who are the members of the EuroBSDcon 
Foundation board? 

JvN: The EuroBSDcon Foundation board currently has 8 
members. Erwin Lansing (member of the FreeBSD Foun- 
dation), S. P. Zeidler (member of the NetBSD Founda- 
tion), Henning Brauer (representing the OpenBSD com- 
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munity), Pawel Jakub Dawidek (organizer of EuroBSDcon 
2012), Mitja Muzenic (candidate for EuroBSDcon 2013) 
and Fred Donck, Paul Schenkeveld and me (organizers 
of EuroBSDcon 2011). 

By organizing the board this way we hope to make sure 
each BSD project is treated equally and that experience is 
passed on between years. 


How can people help the EuroBSDcon 
Foundation? 
JvN: We are, of course, always looking for (international) 
sponsor contacts. Having a network of potential sponsors 
for the coming EuroBSD conferences would make it easi- 
er to organize them in the future. It can also give sponsors 
more visibility during the year, e.g. their logo can be on 
the website when it is announced instead of being visible 
a few months before the conference itself. 

Another way people might help is by donating some of 
their spare time. For example by volunteering to help de- 
sign the website for the EuroBSDcon Foundation. 


How can our readers get involved if they want 
to donate time? 
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JvN: A few things come to mind. We would like to see 
someone with great graphical skills to help design a logo 
for the Foundation and when needed by local organizers 
for that year’s EuroBSDcon. 

Another area we could use help is with marketing. Most 
non-local promotion is now done via the mailing lists and BS- 
Dmag. It would be great if we could have someone promote 
EuroBSDcon in every country of Europe and reach out to 
more BSD enthusiasts. And of course having someone to 
look into organizing the EuroBSDcon in their own country. 

Contact us with ideas and input by sending an email to 
info@eurobsdconfoundation.org. 


Are there any particular skills they should have 

to be able to help? Are there any requirements? 
JvN: The most important qualities are being enthusiastic 
about BSD and willing to learn with us. 


What do volunteers gain in exchange? I’m sure 
that things like fun and experience comes first, 
but maybe also some good connections or 
references that can be helpful as well? 

JvN: Helping with the EuroBSDcon indeed helps you gain 
experience. One of the important thing you learn is to 
work and communicate with people from different cultures 
and countries. This is a plus, because having good com- 
munication skills is a huge advantage when looking for a 
job in the IT industry. 


How do you select the topic and speakers for 
the conference? Is there a chance for young 
geeks as well? Are there any requirements that 
needs to be met in order to be accepted as a 
speaker? 

JvN: The tutorials and talks are selected by the program 
committee, so this is not directly decided by the EuroB- 
SDcon Foundation. The main criteria are the quality of 
the talk or tutorial proposal and the room available in the 
schedule. In an ideal situation the best talks and tutorials 
are selected while maintaining a balance between talks 
about the different BSDs. In reality, this might be a little 
harder to realize due to budget constraints and the prices 
of airplane tickets. For example sometimes a choice has 
to be made to have one superb talk from Australia or 4 
very good talks from different european countries. 

Due to the focus on the quality, | would say the chances 
for young geeks are as good as for the most seasoned 
speakers. Therefore, | would like to ask young geeks not 
to hesitate about sending in their talk proposals, because 
having more choice among talks and getting more people 
involved can only benefit the BSDs. 
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We are also looking into how to improve the selection 
criteria and make them more transparent. As a result, we 
are planning a review of the selection process and how 
we can improve it shortly after EuroBSDcon 2012. 


When do you start planning for the next 
conference? Just after one is finished? What is 
involved in the planning? 

JvN: Ideally we announce the location of the next EuroB- 
SDcon at the closing session of the current. The idea we 
have to make this attainable is to have 2 candidate or- 
ganizers for the next year as a member of the founda- 
tion board during this year. That way the candidates can 
see how organizing a EuroBSDcon works and make an 
informed decision whether they will be able to host the 
next EuroBSDcon. 


Is EuroBSDcon Foundation going to bea 
“possible contact” for the BSD community and 
lovers? 

JvN: Currently the main role we can provide in that re- 
gard is to be a known contact point for questions regard- 
ing the EuroBSDcon during the years to come. Especially 
we think this might make it much easier for international 
sponsors to keep being a sponsor, without the hassle try- 
ing to find the contact for the next EuroBSDcon. 


Have you considered that such a foundation 
can have a more “political” goal? | mean, like 
creating a“BSD lobbying group” in Europe, as it 
is with OS for example. 

JvN: | think it is too early to focus on things like that. The 
main focus we have is to make the organization of the Eu- 
roBSDcon easier. If we start to focus on too many goals, 
we might not reach our main goal. 


Are you planning to extend the activity of the 
Foundation or will it always be dedicated to 
only support EuroBSDcon? What are your plans 
for the future? Where are you heading? 

JvN: Our current and primary focus is the EuroBSDcon. 
One of the things we discussed is that we are willing to 
provide support to other BSD conferences. For example 
by providing our help with the registration system, which 
has been rewritten and can now easily support more than 
one conference. 


Are there any particular flavors of BSD you 
prefer and why? 

JvN: All the BSDs have their strong points. Therefore we 
have a representation of the different major BSDs on the 
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board to ensure that a fair balance between the BSDs, re- 
garding the EuroBSDcon,, is kept. 

Looking at my own situation, | am an advocate of using 
what is useful in your particular situation and of learning 
from each other. For example | am mainly using FreeB- 
SD myself because the jail architecture solves the needs 
| currently have. | would, however, not be able to login to 
them securely without OpenSSH from the OpenBSD proj- 
ect. And without Jorg Sonnenberger’s talk about how Net- 
BSD started using fossil as version management system | 
would not have solved my version management needs as 
| have now. | am also very interested in the way Dragon- 
Fly BSD is going with the HAMMER file system, although 
| still have to look into it more. | also liked the Minix3 (al- 
though not a BSD) talk last year and how they are making 
their OS very robust against failure. 


What are your opinions on how BSD is 
developing? What improvements, if any, would 
you like to see? 

JvN: Technically the BSDs are very strong. One of the 
things that differentiates them from other open source 
operating systems is that good design is put before dirty 
hacks, which makes them very reliable. Being reliable and 
stable, however, does not make you the most popular. Be- 
ing relatively unknown is a disadvantage when suggesting 
to non-BSD aware managers that BSD might solve a par- 
ticular problem. And BSD has some other problems in this 
non-technical area which are hard to address. 

Looking at the technical side, a kickstart or autoyast like 
installation infrastructure might be a huge win for mass in- 
Sstallations. One idea that comes to mind is BSD support in 
spacewalk (http://spacewalk.redhat.com). Improvements 
in HA capabilities might also be a win. 


So the EuroBSDcon Foundation is involved in 
organizing the 2012 conference? 

JvN: We are supporting the 2012 organization. The or- 
ganizing it self is done by Pawel Jakub Dawidek and his 
team in Poland. One of the points we want to keep is that 
each country organizes the EuroBSDcon according to 
their own ideas. So for 2012 the EuroBSDcon Foundation 
is providing help with handling international sponsors and 
setting up the registration system. Especially the last part 
proved to be difficult in earlier years. This year, we have 
rewritten the registration system to become more gener- 
ic SO we wont have problems with that in the upcoming 
years. 
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